web scrapper python请求的问题

时间:2017-04-08 05:40:01

标签: python web-scraping python-requests

当我使用这行代码发送开放网站时,我开始使用python中的请求模块在python中编写webscrapper

Q = requests.post(URL,数据有效载荷=)

从burpsuite中捕获的有效载荷

  

ctl00 $ OnlineContent $ ScriptManager1 = ctl00 $ OnlineContent $ Updatepanel1 | ctl00 $ OnlineContent $ btnGetData&安培; __ EVENTTARGET =安培; __ EVENTARGUMENT =安培; __ VIEWSTATE = / wEPDwUKMTQ2ODcyMzU1Mw9kFgJmD2QWAgIBD2QWBgIBDxYCHgNzcmMFF34vaW1hZ2VzL29ubGluZWxvZ28uanBnZAIDDw8WAh4EVGV4dAUoQS5QLlRyYW5zcG9ydCBEZXBhcnRtZW50IE9ubGluZSBTZXJ2aWNlc2RkAhUPZBYCAgUPZBYCZg9kFgICDQ9kFgICIw88KwANAGQYAQUbY3RsMDAkT25saW5lQ29udGVudCRndmRsc3VzD2dkOZ3fJwrMDniuOIr36W / pcIeY9Cs =安培; __ |上一页= kRa_y-H9KkgKR72aupDjyPGsQHMjHvvPoLZYv2z3H - ToWYU-XKGB_6SWaEFcaDLT52iED4axg4raKfE2TQnH0jFMPazeHbavbmWp-nnAnVyw0ReQW9KD -EXzTMpENJl6481bA2&安培; ctl00 $ $ OnlineContent = ddlInput的R& ctl00 $ $ OnlineContent = txtInput XXXXXXX&安培; ctl00 $ $ OnlineContent btnGetData =获取   数据

import requests
from bs4 import BeautifulSoup
import csv
from tqdm import tqdm

print "contacting Server ....."
print "Fetching results...."
for i in tqdm(range(0,64)):
    url="https://aptransport.in/APCFSTONLINE/Reports/VehicleRegistrationSearch.aspx"
    payload={}
    q=requests.post(url,data=payload)
    idno += 1
    s=BeautifulSoup(q.text,'html.parser')
    htmlfile = open("gitamresults.html",'a')
    htmlfile.write(str(s))
    htmlfile.close()

从burpsuite到python程序的上述请求必须提供的有效负载是什么

0 个答案:

没有答案