Question

我正在抓取www.bing.com搜索引擎，默认情况下它会提供10个结果，但如果我们想要更多结果，我们可以在浏览器中将其增加setting。现在我想在python中使用请求模块获得更多结果。要获得 30 结果，我们必须在cookie中设置 {NRSLT：30} 。

我的代码

payload = {'q': "SACHIN", 'go' : 'Submit+Query', 'qs' : 'bs', 'form' : 'QBRE'}

headers = {'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Firefox/31.0'}

req = requests.get("https://www.bing.com/search", headers=headers, params=payload, timeout=10, cookies={'NRSLT' :'30'})

print req.cookies

<<class 'requests.cookies.RequestsCookieJar'>[<Cookie MUID=0EC3618FCAF862C41C6369B0CB4563A5 for .bing.com/>, <Cookie SRCHD=AF=QBRE for .bing.com/>, <Cookie SRCHUSR=AUTOREDIR=0&GEOVAR=&DOB=20151013 for .bing.com/>, <Cookie _EDGE_S=F=1&SID=3A5F411A73356B471C16492572886A84 for .bing.com/>, <Cookie _EDGE_V=1 for .bing.com/>, <Cookie _FS=NU=1 for .bing.com/>, <Cookie _SS=SID=7DD3F0992D964941871C0C26DE81D3D2 for .bing.com/>, <Cookie MUIDB=0EC3618FCAF862C41C6369B0CB4563A5 for www.bing.com/>, <Cookie SRCHUID=V=2&GUID=00B888BB1B93486087FD9FE628581779 for www.bing.com/>]>

在这里，我没有获得关键＆＃34; NRSLT＆＃34;设置在cookies.Where我出错了？

Answer 1

我会使用现有的Cookie来确保该页面接受cookie。对于此任务，您可以使用Session instance隐式处理cookie。

# create a session object
s = requests.Session()
# retrieve a default cookie
s.get("https://bing.com")

# prepare the request parameters
payload = {'q': "SACHIN", 'go' : 'Submit+Query', 'qs' : 'bs', 'form' : 'QBRE'}
headers = {'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Firefox/31.0'}
# set the cookie variable NRSLT in the existing cookie
s.cookies.set("NRSLT", "50")
# sent prepared request to server
req = s.get("https://www.bing.com/search", headers=headers, params=payload, timeout=10)

在请求模块python中添加cookie

1 个答案: