Question

我想解析一个网页（来自lynda.com）并获取所有课程的标题和链接。所以我使用LWP::UserAgent来获取网址，然后我尝试使用以下代码获取标题和链接：

#!/usr/bin/perl

use LWP::UserAgent;
use strict;
use warnings;

my $ua = new LWP::UserAgent;
my $response = $ua->get('http://www.lynda.com/search?q=android');
unless ($response->is_success) {
    die $response->status_line;
}
my $content = $response->decoded_content();
if (utf8::is_utf8($content)) {
    binmode STDOUT,':utf8';
} else {
    binmode STDOUT,':raw';
}

$content =~ s/[\h\v]+/ /g;
$content =~ s/\r|\n//g; 
$content =~ s/<\/span>|<span>//g;

# Links
my @links = ($content =~ m/<a id="course-result-info.*href="(.*)\?.*class="title">/g);

# titles
my @titles = ($content =~ m/<a id="course-result-info.*class="title"> (.*)<\/a> <span class="author">/g);

print join(", ", @titles);
print "\n---------------------------\n";
print join(", ", @links);

但我只获得了最后一个匹配的（即为亚马逊Kindle设备开发应用程序）。

Answer 1

试试这些正则表达式：

my @links = ($content =~ m/<a id="course-result-info[^>]*href="([^"]*)"[^>]*class="title">/g);

# titles
my @titles = ($content =~ m/<a id="course-result-info[^>]*class="title"> ([^<]*)<\/a> <span class="author">/g);

Answer 2

您的正则表达式解析已被破坏，因为您使用的是贪婪的匹配.*而不是非贪婪的.*?。

然而，更好的解决方案是根本不使用正则表达式，而是使用实际的HTML解析器，例如Mojo::DOM，并具有css选择器的全部功能。

以下使用与Mojo::UserAgent一起安装的Mojolicious来下载网页，然后使用Mojo::DOM来解析返回的结果。这个框架有一个很好的8分钟教程Mojocast Episode 5。

#!/usr/bin/perl

use strict;
use warnings;

use Mojo::UserAgent;

my $url = 'http://www.lynda.com/search?q=android';

my $dom = Mojo::UserAgent->new->get($url)->res->dom;

# Process all links
for my $link ($dom->find('a[id^="course-result-info"]')->each) {
    printf "%s\n    %s\n", $link->all_text(), $link->{href};
}

输出：

Android Studio First Look
    http://www.lynda.com/Android-tutorials/Android-Studio-First-Look/143103-2.html?srchtrk=index%3a1%0alinktypeid%3a2%0aq%3aandroid%0apage%3a1%0as%3arelevance%0asa%3atrue%0aproducttypeid%3a2
Android SDK Essential Training
    http://www.lynda.com/Android-tutorials/Android-SDK-Essential-Training/143102-2.html?srchtrk=index%3a1%0alinktypeid%3a2%0aq%3aandroid%0apage%3a1%0as%3arelevance%0asa%3atrue%0aproducttypeid%3a2
Distributing Android Apps
    http://www.lynda.com/Android-tutorials/Distributing-Android-Apps/143101-2.html?srchtrk=index%3a1%0alinktypeid%3a2%0aq%3aandroid%0apage%3a1%0as%3arelevance%0asa%3atrue%0aproducttypeid%3a2
Building and Monetizing Game Apps for Android
    http://www.lynda.com/Android-tutorials/Building-Monetizing-Game-Apps-Android/107169-2.html?srchtrk=index%3a1%0alinktypeid%3a2%0aq%3aandroid%0apage%3a1%0as%3arelevance%0asa%3atrue%0aproducttypeid%3a2
Building a Note-Taking App for Android
    http://www.lynda.com/Android-tutorials/Building-Note-Taking-App-Android/122466-2.html?srchtrk=index%3a1%0alinktypeid%3a2%0aq%3aandroid%0apage%3a1%0as%3arelevance%0asa%3atrue%0aproducttypeid%3a2
Android SDK: Local Data Storage
    http://www.lynda.com/Android-tutorials/Android-SDK-Local-Data-Storage/112584-2.html?srchtrk=index%3a1%0alinktypeid%3a2%0aq%3aandroid%0apage%3a1%0as%3arelevance%0asa%3atrue%0aproducttypeid%3a2
Android 4.1 SDK Jelly Bean New Features
    http://www.lynda.com/Android-tutorials/Android-SDK-41-Jelly-Bean-New-Features/107922-2.html?srchtrk=index%3a1%0alinktypeid%3a2%0aq%3aandroid%0apage%3a1%0as%3arelevance%0asa%3atrue%0aproducttypeid%3a2
Building Mobile Apps with Google Maps Android API v2
    http://www.lynda.com/Android-tutorials/Building-Mobile-Apps-Google-Maps-Android-API-v2/133347-2.html?srchtrk=index%3a1%0alinktypeid%3a2%0aq%3aandroid%0apage%3a1%0as%3arelevance%0asa%3atrue%0aproducttypeid%3a2
Building Android and iOS Applications with Flex
    http://www.lynda.com/Flash-Builder-4-5-tutorials/Building-Android-and-iOS-Applications-with-Flex/80254-2.html?srchtrk=index%3a1%0alinktypeid%3a2%0aq%3aandroid%0apage%3a1%0as%3arelevance%0asa%3atrue%0aproducttypeid%3a2
Flash Professional CS5: Creating a Simple Game for Android Devices
    http://www.lynda.com/Flash-CS5-tutorials/flash-professional-cs5-creating-a-simple-game-for-android-devices/74928-2.html?srchtrk=index%3a1%0alinktypeid%3a2%0aq%3aandroid%0apage%3a1%0as%3arelevance%0asa%3atrue%0aproducttypeid%3a2
Java Essential Training
    http://www.lynda.com/Java-tutorials/Essential-Training/86005-2.html?srchtrk=index%3a1%0alinktypeid%3a2%0aq%3aandroid%0apage%3a1%0as%3arelevance%0asa%3atrue%0aproducttypeid%3a2
Up and Running with Java Applications
    http://www.lynda.com/Android-tutorials/Up-Running-Java-Applications/94344-2.html?srchtrk=index%3a1%0alinktypeid%3a2%0aq%3aandroid%0apage%3a1%0as%3arelevance%0asa%3atrue%0aproducttypeid%3a2
Building Mobile Apps for Multiple Devices with Flash Professional
    http://www.lynda.com/Flash-Professional-CS5-5-tutorials/Building-Mobile-Apps-for-Multiple-Devices-with-Flash-Professional/89049-2.html?srchtrk=index%3a1%0alinktypeid%3a2%0aq%3aandroid%0apage%3a1%0as%3arelevance%0asa%3atrue%0aproducttypeid%3a2
Developing Applications for Amazon Kindle Devices
    http://www.lynda.com/Android-tutorials/Developing-Applications-Amazon-Kindle-Devices/117102-2.html?srchtrk=index%3a1%0alinktypeid%3a2%0aq%3aandroid%0apage%3a1%0as%3arelevance%0asa%3atrue%0aproducttypeid%3a2

Answer 3

由于您已经手动解析HTML，因此您可能希望使用非贪婪的量词，即。 .*?代替.*代替/s，默认情况下使用.修饰符my @links= $content =~ /<a id="course-result-info.*href="(.*?)\?.*?class="title">/sg;与新行不匹配。

{{1}}

perl - 如何使用RegEx获取所有类似匹配的子字符串

3 个答案: