使用DOMDocument和DOMXPath类从网站检索类别

时间:2014-08-27 01:18:24

标签: php html xpath web-scraping domdocument

我有一个PHP代码,可以使用类名称'子标题'从本网站检索类别。但是,输出不显示任何内容。我究竟做错了什么?

PHP代码:

<?php
header('Content-Type: text/html; charset=utf-8');
$grep = new DoMDocument();
@$grep>loadHTMLFile("http://www.alibaba.com/Products",false,stream_context_create(array("http" => array("user_agent" => "any"))));

$finder = new DomXPath($grep);
$class = "sub-title";
$nodes = $finder->query("//*[contains(@class, '$class')]");

foreach ($nodes as $node) {
    $span = $node->childNodes;
    echo $span->item(0)->nodeValue;

}

?>

期望的输出:
农业
食品与食品饮料
服装
等。

谢谢!

2 个答案:

答案 0 :(得分:1)

只针对那个特定元素。顺便说一句,您当前的代码在$grep>loadHTMLFile上有拼写错误。它在-中缺少->。我修改了一下。

$ch = curl_init('http://www.alibaba.com/Products');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT,'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13');
$html = curl_exec($ch);
$dom = new DOMDocument();
@$dom->loadHTML($html);
$finder = new DOMXPath($dom);
$nodes = $finder->query('//h4[@class="sub-title"]');

foreach ($nodes as $node) {
    $sub_title = trim(explode("\n", trim($node->nodeValue))[0]) . '<br/>';
    echo $sub_title;
}

答案 1 :(得分:1)

要在使用DOMDocument::loadHTMLFile获取HTML时设置流上下文,请使用libxml_set_streams_context

<?php

$context = stream_context_create(array('http' => array('user_agent' => 'any')));
libxml_set_streams_context($context);
libxml_use_internal_errors(true);

$doc = new DOMDocument();
$doc->loadHTMLFile('http://www.alibaba.com/Products');

$xpath = new DOMXPath($doc);
$nodes = $xpath->query('//h4[@class="sub-title"]/a');

foreach ($nodes as $node) {
    echo trim($node->textContent) . "\n";
}