我正在编写一个URL抓取工具(只是名称和说明),我正在尝试处理301重定向。
现在,我检查标题,如果不是200,我会尝试找到要重定向到标题内的位置。我的问题出现了,因为尽管我在那里看到了,但是array_search没有返回Location值的键。
这是代码段:
if(strpos($url_headers[0], "200") !== false){
echo "in here";
return $url;
}else{
print_r($url_headers);
//look for location
$location_key = array_search("Location: ", $url_headers);
echo "Location Key: " . $location_key;
$redirect_string = $url_headers[$location_key];
$clean_url = str_replace("Location: ", "", $redirect_string);
return $clean_url;
}
这个输出是:
Array ( [0] => HTTP/1.0 301 Moved Permanently [1] => Location: http://www.google.com/ [2] => Content-Type: text/html; charset=UTF-8 [3] => Date: Wed, 13 Feb 2013 03:30:00 GMT [4] => Expires: Fri, 15 Mar 2013 03:30:00 GMT [5] => Cache-Control: public, max-age=2592000 [6] => Server: gws [7] => Content-Length: 219 [8] => X-XSS-Protection: 1; mode=block [9] => X-Frame-Options: SAMEORIGIN [10] => HTTP/1.0 200 OK [11] => Date: Wed, 13 Feb 2013 03:30:00 GMT [12] => Expires: -1 [13] => Cache-Control: private, max-age=0 [14] => Content-Type: text/html; charset=ISO-8859-1 [15] => Set-Cookie: PREF=ID=fe86e29432d4e240:FF=0:TM=1360726200:LM=1360726200:S=Wg8VEU7kc7UtcKc-; expires=Fri, 13-Feb-2015 03:30:00 GMT; path=/; domain=.google.com [16] => Set-Cookie: NID=67=KH8Zu8EpKjrhje8nD0lk_868mqvQr9pGwsAsaUuPDD_PRUgohJHoOkdlyYEHWmohUtndyENDJ0oZq8pC1aqOg20anXpUn5btQX5GYM6kYlgMhYxIPajtGp9KymmMDO1Y; expires=Thu, 15-Aug-2013 03:30:00 GMT; path=/; domain=.google.com; HttpOnly [17] => P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info." [18] => Server: gws [19] => X-XSS-Protection: 1; mode=block [20] => X-Frame-Options: SAMEORIGIN ) Location Key: {"error":"invalid_url","error_code":null}
我做错了什么?在抓取用户提供的链接时,是否有更优雅的方法来处理重定向?
答案 0 :(得分:0)
$url_headers[0] = 'HTTP/1.0 200';
if(strpos($url_headers[0], "200") > 0){
echo "here";
} else {
//look for location
$location_key = getLocation($url_headers);
echo "Location Key: " . $location_key;
}
function getLocation($data) {
$url = false;
foreach($data as $key => $value) {
if (preg_match("/Location:/", $value)) {
echo "A match was found.";
//$url = $matches[1];
$url = $data[$key];
break;
}
}
return $url;
}
答案 1 :(得分:0)
strpos如果找不到匹配则返回false,因此您需要执行
if( ! strpos($url_headers[0], "200"))
答案 2 :(得分:0)
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$a = curl_exec($ch);
if(preg_match('#Location: (.*)#', $a, $r)){
$l = trim($r[1]);
return $l;
}else{
return $url;
}
这大部分都有效,但仍然无法重定向到https(由于某种原因,他们需要双重定向?)