Question

当我尝试使用CURL解析它时，此站点不返回任何内容。这是我的代码：

/* gets the data from a URL */
function get_data($url) {
    $ch = curl_init();
    $timeout = 5;
    curl_setopt($ch,CURLOPT_URL,$url);
    curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
    curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,$timeout);
    $data = curl_exec($ch);
    curl_close($ch);
    return $data;
}

$returned_content = get_data('http://casting.backstage.com/jobseekerx/SearchJobs.asp?SubmitToSearch=Search&lctr=1&rvsd=-1&o1=2&p1=1&ipp=10&city=&fromsearchpage=true&cg=11&cg=12&cg=13&cg=14&cg=15&cg=16&cg=17&cg=18&cg=19&cg=20&cg=22&kwrd=&kwdt=1&lcta=1&btnSearch=Run+Search+Now');

print $returned_content;

我从未遇到过这个问题而且我一直都在使用这种方法。我也尝试过使用Simple DOM Parser并获得相同的结果。这是有问题的网址：

http://casting.backstage.com/jobseekerx/SearchJobs.asp?SubmitToSearch=Search&lctr=1&rvsd=-1&o1=2&p1=1&ipp=10&city=&fromsearchpage=true&cg=11&cg=12&cg=13&cg=14&cg=15&cg=16&cg=17&cg=18&cg=19&cg=20&cg=22&kwrd=&kwdt=1&lcta=1&btnSearch=Run+Search+Now

此页面上是否有某种反爬虫代码？

Answer 1

这可能是因为生成的HTML总是垃圾：

http://validator.w3.org/check?uri=http%3A%2F%2Fcasting.backstage.com%2Fjobseekerx%2FSearchJobs.asp%3FSubmitToSearch%3DSearch%26lctr%3D1%26rvsd%3D-1%26o1%3D2%26p1%3D1%26ipp%3D10%26city%3D%26fromsearchpage%3Dtrue%26cg%3D11%26cg%3D12%26cg%3D13%26cg%3D14%26cg%3D15%26cg%3D16%26cg%3D17%26cg%3D18%26cg%3D19%26cg%3D20%26cg%3D22%26kwrd%3D%26kwdt%3D1%26lcta%3D1%26btnSearch%3DRun%2BSearch%2BNow&charset=%28detect+automatically%29&doctype=Inline&group=0

Answer 2

你看到你的错误了吗？ echo curl_error($ch)可让您查看您遇到的错误究竟是什么。在此基础上，您可以在许多情况下继续解决问题。在这种特殊情况下，我添加了一个CURLOPT_USERAGENT字段，它运行良好。

<?php
function get_data($url) 
{
    $ch = curl_init();
    $timeout = 30;
    curl_setopt($ch,CURLOPT_URL,$url);
    curl_setopt($ch,CURLOPT_RETURNTRANSFER,false);
    curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,$timeout);
    curl_setopt($ch,CURLOPT_POST,false);
    curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:10.0) Gecko/20100101 Firefox/10.0");
    $data = curl_exec($ch);
    curl_close($ch);

    return $data;
}
$url = 'http://casting.backstage.com/jobseekerx/SearchJobs.asp?SubmitToSearch=Search&lctr=1&rvsd=-1&o1=2&p1=1&ipp=10&city=&fromsearchpage=true&cg=11&cg=12&cg=13&cg=14&cg=15&cg=16&cg=17&cg=18&cg=19&cg=20&cg=22&kwrd=&kwdt=1&lcta=1&btnSearch=Run+Search+Now';
$returned_content = get_data($url);
print $returned_content;
?>

我希望这会对你有所帮助。

PHP：无法使用CURL解析特定站点

2 个答案: