Question

我有以下curl命令：

curl -H "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0" -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" -H "Connection: keep-alive" -X GET http://example.com/en/number/111555000

不幸的是我无法复制它...... 我尝试过：

   url = http://example.com/en/number/111555000
   headers = {'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0', 'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'Connection':'keep-alive',}
   req = urllib2.Request(url, None, headers)
   resp = urllib2.urlopen(req)
   print resp.read()

但服务器识别出该请求是如何“假”并将我转发给谷歌（来自服务器的回复是：HTTP/1.1 301 Moved Permanently）。使用curl而不是我收到原始页面。

有任何想法或建议吗？谢谢 DK

编辑：一些额外的信息：

$ nc example.com 80 
GET /en/number/111555000 HTTP/1.1
Host: example.com

HTTP/1.1 301 Moved Permanently
Date: Fri, 29 May 2015 18:51:05 GMT
Server: Apache
X-Powered-By: PHP/5.5.24
Location: http://www.google.de
Content-Length: 0
Content-Type: text/html


$ nc example.com 80 
GET /en/number/111555000 HTTP/1.1
Host: example.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Connection: keep-alive

HTTP/1.1 200 OK
Date: Fri, 29 May 2015 18:57:56 GMT
Server: Apache
X-Powered-By: PHP/5.5.24
Set-Cookie: session=a%3A4%3A%7Bs...
Set-Cookie: session=a%3A4%3A%7Bs...
Keep-Alive: timeout=2, max=200
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8

1c6f8
<!DOCTYPE html>
[...]

卷曲：

$curl -X GET http://example.com/en/number/111555000
$

$ curl -H "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0" -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" -H "Connection: keep-alive" -X GET http://example.com/en/number/111555000
<!DOCTYPE html>
[...]

Answer 1

我可以使用它来处理请求库。这是probably better to use.

import requests
url = "http://example.com/en/number/111555000"
headers = {'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0', 'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'Connection':'keep-alive',}
req = requests.get(url, headers=headers)
req.text

here is the requests library documentation

希望它有所帮助。

Python http get - 无法使用标题

1 个答案: