PHP使用CURL抓取网页时处理cookie

时间:2017-06-18 18:51:11

标签: php curl cookies web-scraping

我正在尝试抓取页面,但由于Cookie,我因许多页面而失败。我想我需要抓取页面的cookie并将它们放入CURLOPT_COOKIE。但我仍然无法获得此类网页的内容。

CURL的网址:http://www.roguefitness.com/reebok-crossfit-speed-teal-dust-forest-grey-black-hery-yellow-skull-grey

$cookie = array();
$cookie[] = "visid_incap_626072=wppfexB2Rx2G+0Fl0p5ObF3HRlkAAAAAQUIPAAAAAABkbU4BWHOCgsesm58ql+4y";
$cookie[] = "incap_ses_415_626072=L8pAJA2442e3l2pfKmHCBV3HRlkAAAAAK0T9IPzsPD+zRr2W8AY4kA==";
$cookie[] = "incap_ses_541_626072=BGgpDAFx32HueJcV/QSCB5nHRlkAAAAAysy38ULGSs0RqTsyx2QsJg==";
$cookie = implode('; ', $cookie);

$cookieJar = tempnam ("/dev/null", "CURLCOOKIE");
$cookieFile = tempnam ("/dev/null", "CURLCOOKIE");      
curl_setopt( $ch, CURLOPT_COOKIEJAR, $cookieJar );
curl_setopt( $ch, CURLOPT_COOKIEFILE, $cookieFile );
curl_setopt($ch, CURLOPT_COOKIE, $cookie);

响应不是页面内容,而是:

HTTP/1.1 200 OKContent-Type: text/htmlCache-Control: no-cacheConnection: closeContent-Length: 670X-Iinfo: 8-65878311-0 0NNN RT(1497811103027 0) q(0 -1 -1 -1) r(0 -1) B12(4,316,0)

真的很感激任何帮助!

1 个答案:

答案 0 :(得分:0)

这是缺少使用它的代码行:

curl_setopt($ch, CURLOPT_COOKIESESSION, true);