如何在页面上的<a href="">SCRAPE THIS</a>等所有链接标记之间抓取内容?

时间:2012-08-23 20:33:57

标签: php xpath screen-scraping

我正在试图抓取网站的链接文本,即废话。我想对页面上的所有链接执行此操作。到目前为止,我有这个:

<?php

$target_url = "SITE I WANT TO SCRAPE";

// make the cURL request to $target_url
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$target_url);
curl_setopt($ch, CURLOPT_FAILONERROR, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,true);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
$html= curl_exec($ch);
if (!$html) {
    echo "<br />cURL error number:" .curl_errno($ch);
    echo "<br />cURL error:" . curl_error($ch);
    exit;
}

// parse the html into a DOMDocument
$dom = new DOMDocument();
@$dom->loadHTML($html);

// grab all the on the page
$xpath = new DOMXPath($dom);
$hrefs = $xpath->evaluate("/html/body//a/text()");

for ($i = 0; $i < $hrefs->length; $i++) {
    $href = $hrefs->item($i);
    echo "<br />Link stored: $href";
}
?>

我对这些东西很陌生,无法弄清楚我做错了什么?

谢谢!

2 个答案:

答案 0 :(得分:2)

在for循环中,$href不是字符串。它实际上是一个DOMText节点。要将其用作字符串,您需要访问其nodeValue属性。

for ($i = 0; $i < $hrefs->length; $i++) {
    $href = $hrefs->item($i);
    echo "<br />Link stored: $href->nodeValue";
}

答案 1 :(得分:1)

尝试:

$xpath = new DOMXPath($dom);
$hrefs = $xpath->evaluate("/html/body//a/text()");

for ($i = 0; $i < $hrefs->length; $i++) {
    $href = $hrefs->item($i)->textContent;
    echo "<br />Link stored: $href";
}