Question

我试图使用此代码php在betfair.com网站上进行网页抓取：

<?php    
    // Defining the basic cURL function
    function curl($url) {
        $ch = curl_init();  // Initialising cURL
        curl_setopt($ch, CURLOPT_URL, $url);    // Setting cURL's URL option with the $url variable passed into the function
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); // Setting cURL's option to return the webpage data
        $data = curl_exec($ch); // Executing the cURL request and assigning the returned data to the $data variable
        curl_close($ch);    // Closing cURL
        return $data;   // Returning the data from the function
    }    

    $scraped_website = curl("https://www.betfair.com/exchange/football");       
    echo $scraped_website;          
?>

以这种方式运行的代码。

但是，如果不是＆＃34; https://www.betfair.com/exchange/football＆＃34;选择＆＃34; https://www.betfair.com/exchange/football/event?id=28040884＆＃34; 代码停止工作。

请帮助。

Answer 1

查看curl收到的标题：

 HTTP/1.1 302 Moved Temporarily
 Location: https://www.betfair.com/exchange/plus/#/football/event/28040884
 Cache-Control: no-cache
 Pragma: no-cache
 Date: Fri, 09 Dec 2016 17:38:52 GMT
 Age: 0
 Transfer-Encoding: chunked
 Connection: keep-alive
 Server: ATS/5.2.1
 Set-Cookie: vid=00956994-084c-444b-ad26-38b1119f4e38; Domain=.betfair.com; Expires=Mon, 01-Dec-2022 09:00:00 GMT; Path=/
 X-Opaque-UUID: 80506a77-12c1-4c89-b4a6-fa499fd23895

实际上https://www.betfair.com/exchange/football/event?id=28040884发送了302 Moved Temporarily HTTP重定向，并且您的脚本不遵循重定向，这就是它无法正常工作的原因。修复（使用CURLOPT_FOLLOWLOCATION），你的代码工作正常。固定代码：

function curl($url) {
    $ch = curl_init();  // Initialising cURL
    curl_setopt($ch, CURLOPT_URL, $url);    // Setting cURL's URL option with the $url variable passed into the function
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); // Setting cURL's option to return the webpage data
    curl_setopt($ch, CURLOPT_VERBOSE, TRUE);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
    $data = curl_exec($ch); // Executing the cURL request and assigning the returned data to the $data variable
    curl_close($ch);    // Closing cURL
    return $data;   // Returning the data from the function
}
var_dump(curl("https://www.betfair.com/exchange/football/event?id=28040884"));

（我还建议使用CURLOPT_ENCODING =＆gt;＆＃39;＆＃39;，如果支持curl将使用压缩传输，并且HTML压缩确实非常好，使用gzip通常编译为支持卷曲，使网站下载速度更快，这使得curl_exec（）返回得更快）

php cUrl在betfair上受阻？

1 个答案: