PHP / cURL脚本的替代方案:Web自动化?

时间:2011-12-01 15:48:04

标签: php curl

我正在尝试使用PHP和cURL创建一个脚本,该脚本可以提取this页面中显示的电子邮件。

我遇到问题,因为酒店列表是通过ajax调用加载的。

好吧,我的问题是:PHP / cURL脚本有哪些替代方案?也许某些工具如Selenium,Sahi或Watir(这些是用于测试的)可以打开浏览器???

Linux用户。

修改

在牧师回答之后,我想在这里粘贴我的代码,我将获得一个xml文件,其中包含第一页酒店的数据,但我想访问其他酒店的页面是......怎么做?

<?php

//extract data from the post
extract($_POST);

//set POST variables
$url = 'http://www.turismovenezia.it/index.php';

$fields1 = array(
            'ajax'=>'searchEngineTopdata',
            'next_pair'=>'Dove Allogiare|*',
            'lang'=>'it');


$fields2 = array(

'ajax'=>'xmlSearchEngineResponder',
'xml' => "%3C%3Fxml%20version%3D%221.0%22%3F%3E%3CSearchRequest%20xmlns%3D%22http%3A%2F%2Fwww.liberologico.com%2Fdbsite%2Fjolly-search%22%20xmlns%3Axsi%3D%22http%3A%2F%2Fwww.w3.org%2F2001%2FXMLSchema-instance%22%3E%3CSearch%3E%3CScope%3E%3C%21%5BCDATA%5B%2A%5D%5D%3E%3C%2FScope%3E%3CFilters%3E%3CFilters%20xsi%3Atype%3D%22FilterSpecType%22%3E%3CField%3Eaptve_territorio%3C%2FField%3E%3CValue%3E%3CSingleValue%3E%3C%21%5BCDATA%5B%2A%5D%5D%3E%3C%2FSingleValue%3E%3C%2FValue%3E%3CMode%3ETHESAURUS%3C%2FMode%3E%3COperation%3ELIKE%3C%2FOperation%3E%3C%2FFilters%3E%3CFilters%20xsi%3Atype%3D%22FilterSpecType%22%3E%3CField%3Efull_text_search%3C%2FField%3E%3CValue%3E%3CSingleValue%3E%3C%21%5BCDATA%5B%2A%5D%5D%3E%3C%2FSingleValue%3E%3C%2FValue%3E%3CMode%3EFREE_TEXT%3C%2FMode%3E%3COperation%3ELIKE%3C%2FOperation%3E%3C%2FFilters%3E%3CFilters%20xsi%3Atype%3D%22FilterSpecType%22%3E%3CField%3Elang%3C%2FField%3E%3CValue%3E%3CSingleValue%3E%3C%21%5BCDATA%5Bit%5D%5D%3E%3C%2FSingleValue%3E%3C%2FValue%3E%3CMode%3EFREE_TEXT%3C%2FMode%3E%3COperation%3EEQUAL%3C%2FOperation%3E%3C%2FFilters%3E%3C%2FFilters%3E%3CSubSearches%3E%3CSearch%3E%3CScope%3E%3C%21%5BCDATA%5BEventi%5D%5D%3E%3C%2FScope%3E%3C%2FSearch%3E%3CSearch%3E%3CScope%3E%3C%21%5BCDATA%5BArte%20%26%20Cultura%5D%5D%3E%3C%2FScope%3E%3C%2FSearch%3E%3CSearch%3E%3CScope%3E%3C%21%5BCDATA%5BMare%20%26%20Natura%5D%5D%3E%3C%2FScope%3E%3C%2FSearch%3E%3CSearch%3E%3CScope%3E%3C%21%5BCDATA%5BPiatti%20%26%20Prodotti%20tipici%5D%5D%3E%3C%2FScope%3E%3C%2FSearch%3E%3CSearch%3E%3CScope%3E%3C%21%5BCDATA%5BRelax%20%26%20Divertimento%5D%5D%3E%3C%2FScope%3E%3C%2FSearch%3E%3CSearch%3E%3CScope%3E%3C%21%5BCDATA%5BDove%20Alloggiare%5D%5D%3E%3C%2FScope%3E%3C%2FSearch%3E%3CSearch%3E%3CScope%3E%3C%21%5BCDATA%5BDove%20Mangiare%5D%5D%3E%3C%2FScope%3E%3C%2FSearch%3E%3CSearch%3E%3CScope%3E%3C%21%5BCDATA%5BInformazioni%20Utili%5D%5D%3E%3C%2FScope%3E%3C%2FSearch%3E%3C%2FSubSearches%3E%3C%2FSearch%3E%3CActiveResultSet%3E%3CTab%3E%3C%21%5BCDATA%5BDove%20Alloggiare%5D%5D%3E%3C%2FTab%3E%3CFirstItem%3E0%3C%2FFirstItem%3E%3CPagerSize%3E10%3C%2FPagerSize%3E%3C%2FActiveResultSet%3E%3C%2FSearchRequest%3E",
'force' => 'false');

//open connection
$ch = curl_init();

//set the url, number of POST vars, POST data
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_POST, true);
curl_setopt($ch,CURLOPT_POSTFIELDS, $fields1);
curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);

//set the url, number of POST vars, POST data
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_POST, true);
curl_setopt($ch,CURLOPT_POSTFIELDS, $fields2);
curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);

//execute post
$result = curl_exec($ch);

echo $result;

//close connection
curl_close($ch);

3 个答案:

答案 0 :(得分:2)

你可能不会接受这个答案,但也许社区会这样。您需要听到的答案是:不要从网页中提取电子邮件。

无论您想要推广到这些地址,您都不希望向他们发送未经请求的电子邮件。这将惹恼那些人,并且会导致您的电子邮件被提交到SpamCop或其他垃圾邮件黑名单。这也是非常粗鲁的行为。

不,如果电子邮件是“仅限一次”或者您提供“退出说明”,则无关紧要。

尝试联系网站本身以获取帮助。他们可能有一个论坛供您发布消息,或者他们可能代表您转发(不实际向您提供电子邮件数据库)。

答案 1 :(得分:2)

该列表是由http://www.turismovenezia.it/index.php的POST请求提取的XML列表。 cURL是IMO,是您采取此行动的最佳工具。使用开发人员的控制台。了解网站如何启动并完成请求,然后模仿它。

RAW POST using cURL in PHP

答案 2 :(得分:-1)

查看simple_html_dom库

http://simplehtmldom.sourceforge.net/

加载DOM并使用PHP操作的简单方法