我正在尝试获取此页面的完整HTML内容:http://www.ims.com.br/ims/visite/programacao/2017/03/05
但是,HTML代码的一部分来自以下的AJAX请求:http://www.ims.com.br/ajax/programacao.php¹
根据Chrome的Web Inspector的要求,该请求具有以下标题:
POST /ajax/programacao.php HTTP/1.1
Host: www.ims.com.br
Connection: keep-alive
Content-Length: 51
Accept: text/html, */*; q=0.01
Origin: http://www.ims.com.br
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.76 Safari/537.36
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Referer: http://www.ims.com.br/ims/visite/programacao/2017/03/05
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.8
Cookie: PHPSESSID=k3cu5ho6ev7g9j56jpoosqth94; _ga=GA1.3.252384583.1488689974
以下表格数据:
date=1488682800&visualizacao=hoje&onde=&oque=&como=
其中Date是时间戳。
但是,当我使用以下代码在Python中执行request.post时,来自页面¹的HTML未完成。
#!/usr/bin/env python
# coding: utf-8
# Instituto Moreira Salles
import os
import re
import time
import requests
def main():
numday = time.strftime("%w")
if(numday == '1'):
timestamp = str(int(time.time()) - 86400)
else:
timestamp = str(int(time.time()))
payload = {'date': timestamp, 'visualizacao': 'hoje', 'onde' : '', 'oque' : '', 'como' : ''}
header = {'accept': 'text/html, */*; q=0.01',
'accept-encoding': 'gzip, deflate',
'accept-language': 'en-US,en;q=0.8',
'connection': 'keep-alive',
'content-length': '51',
'content-type': 'application/x-www-form-urlencoded',
'cookie': 'PHPSESSID=k3cu5ho6ev7g9j56jpoosqth94; _gat=1; _ga=GA1.3.252384583.1488689974; PHPSESSID=2jmgciitgoub7o74up18rqikl6',
'origin': 'http://www.ims.com.br',
'referer': 'http://www.ims.com.br/ims/visite/programacao/2017/03/05',
'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.76 Safari/537.36',
'x-requested-with': 'XMLHttpRequest'}
hhtml = requests.post('http://www.ims.com.br/ajax/programacao.php', data=payload), headers=header)
print hhtml.content
main()
但如果我在Chrome扩展应用“高级REST客户端”中执行相同的请求,则HTML内容已完成。
任何人都知道如何使用Python中的标题和数据进行正确的POST请求?