如何在html树中移动节点并提取链接?

时间:2012-09-13 07:46:22

标签: perl html-parsing

我知道我的问题标题不具有描述性,但让我在这里解释一下。

我正在尝试使用HTML :: TreeBuilder解析给定的html document。现在在这个html文档中,值5,1,ABC,DEF将根据用户提供的值进行验证,如果验证成功,我必须提取href链接。

所以,我的代码是:

my @tag = $tree->look_down( _tag => 'tr', class => qr{\bepeven\scompleted\b} );
for (@tag) {

    query_element($_);
}

sub query_element {

    my @td_tag = $_[0]->look_down( _tag => 'td' );

    my $num1 = shift @td_tag; #Get the first td tag
    my $num2 = shift @td_tag; # Get the second td tag


    #Making sure first/second td tag has numeric value
    $num1 = $1 if $num1->as_text =~ m!(\d+)! or die "no match found";
    $num2 = $1 if $num2->as_text =~ m!(\d+)! or die "no match found";


    #Validating that above value's match the user provided value 5 and 1.
    if ( $num1 eq '5' && $num2 eq '1' ) { 
        say "hurray..!!";

        #Iterating over rest of the td tag to make sure we get the right link from it.
        for (@td_tag) {

            #Check if contains ABC and than procede to fetch the download href link.
            if ($_->look_down(_tag  => 'td', class => qr{[c]}, sub {
                        $_[0]->as_text eq 'ABC';} )
            )   
            {   
                my $text = $_->as_text;
                say "Current node text is: ", $text; #outputs ABC

                #Now from here how do I get the link I want to extract.
            }
        }
    }
}

现在,我的方法首先从td tags中提取值,并将其与用户指定的值匹配,如果它成功,则查找另一个用户指定的值ABC or DEF在我的情况下它是{ {1}}如果匹配而非提取链接。

现在,标记containsig ABC没有固定的位置,但它们将位于包含ABC or DEF值的标记之下。所以,我使用5 and 1来表示标签包含$_[0]->as_text eq 'ABC';现在在我的树中我目前在ABC ABC从这里如何提取链接href i,我该如何移动提取对象树并提取值。

PS:我会在这里试过xpath但是html元素的位置不是那么定义和结构化。

修改

所以,我尝试了text node并返回了$_->tag(),但是如果我使用的是td标记,那么以下代码无效:

td

但它会出现以下错误:

my $link_obj = $_->look_down(_tag => 'a') # It should look for `a` tag.
say $link_obj->as_text;

3 个答案:

答案 0 :(得分:4)

我希望以下(使用我自己的Marpa :: R2 :: HTML)很有帮助。注意 HTML :: TreeBuilder答案只找到一个答案。下面的代码找到 二,我认为是意图。

#!perl

use Marpa::R2::HTML qw(html);

use 5.010;
use strict;
use warnings;

my $answer = html(
    ( \join q{}, <DATA> ),
    {   td => sub { return Marpa::R2::HTML::contents() },
        a  => sub {
            my $href = Marpa::R2::HTML::attributes()->{href};
            return undef if not defined $href;
            return [ link => $href ];
        },
        'td.c' => sub {
            my @values = @{ Marpa::R2::HTML::values() };
            if ( ref $values[0] eq 'ARRAY' ) { return $values[0] }
            return [ test => 'OK' ] if Marpa::R2::HTML::contents eq 'ABC';
            return [ test => 'OK' ] if Marpa::R2::HTML::contents eq 'DEF';
            return [ test => '' ];
        },
        tr => sub {
            my @cells = @{ Marpa::R2::HTML::values() };
            return undef if shift @cells != 5;
            return undef if shift @cells != 1;
            my $ok = 0;
            my $link;
            for my $cell (@cells) {
                my ( $type, $value ) = @{$cell};
                $ok = 1 if $type eq 'test' and $value eq 'OK';
                $link = $value if $type eq 'link';
            }
            return $link if $ok;
            return undef;
        },
        ':TOP' => sub { return Marpa::R2::HTML::values(); }
    }
);

die "No parse" if not defined $answer;
say join "\n", @{$answer};

__DATA__
<table>
    <tbody>

        <tr class="epeven completed">
            <td>5</td>
            <td>1</td>
            <td class="c">ABC</td>
            <td class="c">satus</td>
            <td class="c"><a href="/path/link">Download</a></td>
        </tr>
        <tr class="epeven completed">
            <td>5</td>
            <td>1</td>
            <td class="c">status</td>
            <td class="c">DEF</td>
            <td class="c"><a href="/path2/link">Download</a></td>
        </tr>


    </table>

答案 1 :(得分:2)

我不确定我理解你的目标是什么,但这些内容是什么?使用look_down描述你想要的东西,没有必要尝试在树上导航自己;那将是脆弱的。

use strict;
use warnings;
use HTML::TreeBuilder 5 -weak;
use 5.014;

my $tree = HTML::TreeBuilder->new_from_content(<DATA>);


for my $e ($tree->look_down( _tag => 'a',
                             sub { my $e = $_[0];
                                   my $tr = $e->parent->parent; ### Could also use ->lineage to search up through the 
                                                                ### containing elements
                                   return unless $tr->attr('_tag') eq 'tr' and $tr->attr('class') eq 'epeven completed';
                                   return (     $tr->look_down( _tag => 'td', sub { $_[0]->as_text eq '1'; })
                                            and $tr->look_down( _tag => 'td', sub { $_[0]->as_text eq '5'; })
                                            and $tr->look_down( _tag => 'td', class => 'c', sub { $_[0]->as_text eq 'ABC'; })
                                          );
                                 }
                           )
          ) {
    say $e->attr('href');
}


__DATA__

<table>
    <tbody>

        <tr class="epeven completed">
            <td>5</td>
            <td>1</td>
            <td class="c">ABC</td>
            <td class="c">satus</td>
            <td class="c"><a href="/path/link">Download</a></td>
        </tr>
        <tr class="epeven completed">
            <td>5</td>
            <td>1</td>
            <td class="c">status</td>
            <td class="c">DEF</td>
            <td class="c"><a href="/path2/link">Download</a></td>
        </tr>


    </table>

输出:

/path/link

答案 2 :(得分:0)

如果你可以放弃HTML :: TreeBuilder,你可能会解析这样的事情:

for my $r ($content =~ m{<tr class="epeven completed">(.*?)</tr>}gs) {
    my ($n1, $n2) = $r =~ m{<td>(\d+)</td>\s*<td>(\d+)</td>}g;
    next if $n1 != 5 || $n2 != 1;
    next if $r !~ m{<td class="c">ABC</td>}g;
    my ($link) = $r =~ m{<a href="(.*?)">Download</a>}g;
    say $link;
}