Question

我一直在使用Mechanize :: Firefox来进行网络抓取，而我目前正处于完全无头的状态。我一直在使用这段代码：

my $link = [$firefox->find_link_dom(text_regex => qr/pdf/i)]->[0]->{href} . "\n";

在查找.pdf文件方面提供了非常好的结果。无论如何我可以使用不需要浏览器显示的不同模块找到基于text_regex的链接厄运吗？

修改

使用Mechanize :: Firefox，我会从代码中找回一个链接：

<div id="rhc" xpathLocation="noDialog">

 <div id="download" class="rhcBox_type1">
<div class="wrap">
  <ul>
    <li class="download icon"><strong>Download:</strong>
      <a href="/article/fetchObjectAttachment.action;jsessionid=CDBD1AACEEB3CF89729EB23808FD1319?uri=info%3Adoi%2F10.1371%2Fjournal.pbio.1001193&amp;representation=PDF" title="Download article PDF">PDF</a> |
      <a href="/article/citationList.action;jsessionid=CDBD1AACEEB3CF89729EB23808FD1319?articleURI=info%3Adoi%2F10.1371%2Fjournal.pbio.1001193" title="Download citations">Citation</a> |
      <a href="/article/fetchObjectAttachment.action;jsessionid=CDBD1AACEEB3CF89729EB23808FD1319?uri=info%3Adoi%2F10.1371%2Fjournal.pbio.1001193&amp;representation=XML" title="Download Article XML">XML</a>
    </li>
    <li class="print icon"><a href="#" onclick="window.print();return false;" title="Print Article"><strong>Print article</strong></a></li>
    <li class="reprint icon"><a href="https://www.odysseypress.com/onlinehost/reprint_order.php?type=A&page=0&journal=1&doi=10.1371/journal.pbio.1001193&volume=&issue=&title=Evolutionarily Conserved Linkage between Enzyme Fold, Flexibility, and Catalysis&author_name=Arvind%20Ramanathan%2C%20Pratul%20K.%20Agarwal&start_page=1&end_page=17" title="Odyssey Press">EzReprint</a> New &amp; improved!</li>
  </ul>
</div>

下载pdf文件。任何想法为什么HTML :: Treebase不会得到这个链接??

Answer 1

CSS3 a[href*="pdf"]：Web::Query，HTML::Query，Mojo::UserAgent
XPath //a[contains(@href, "pdf")]：HTML::TreeBuilder::XPath，XML::LibXML

Answer 2

use strict;
use warnings;
use HTML::TreeBuilder::XPath qw(); 

my $dom = HTML::TreeBuilder::XPath->new_from_content(q(...)); # ... your html code
my @nodes = $dom->findnodes('//a[contains(@href, "pdf")]'); 

foreach my $node (@nodes) {
  my $href = $node->findvalue('@href');
  ...
}

Perl通过DOM查找Web链接

2 个答案: