如何使用请求模块从Web下载大数据文件?

时间:2016-06-21 09:43:01

标签: python download request

我正在使用python从网页下载文件,在我提交请求后,我得到如下标题:

r.headers=
{'Content-Disposition': 'attachment; filename="report20160619-013623.csv";',
 'Content-Transfer-Encoding': 'binary', 'Expires': '0',
'Keep-Alive': 'timeout=5, max=100', 'Server': 'Apache',
'Transfer-Encoding': 'chunked', 'Connection': 'Keep-Alive', 'Pragma': 'public',
'Cache-Control': 'must-revalidate, post-check=0, pre-check=0, private',
'Date': 'Sun, 19 Jun 2016 06:35:18 GMT', 'X-Frame-Options': 'SAMEORIGIN',
'Content-Type': 'application/octet-stream'}

我可以直接从网页下载大约50兆字节的文件,但是当我使用python请求时,它不起作用。它返回:

r = sss.get(url, stream=True, headers=my_headers)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests-2.8.1-py2.7.egg/requests/sessions.py", line 480, in get
    return self.request('GET', url, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests-2.8.1-py2.7.egg/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests-2.8.1-py2.7.egg/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests-2.8.1-py2.7.egg/requests/adapters.py", line 426, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine("''",))

这是我的网址: https://aosreports2.corp.XXX.com/DOSSIER/?period=Date+Range&report_period=Current&start_date=2016-02-28&end_date=2016-03-15&data=bookings&low_val=&high_val=&search_var=none&search_vals=&run=Download&m%5B%5D=Gross+USD+Value&m%5B%5D=Attach+BTB+USD&m%5B%5D=Net+Units&r%5B%5D=Channel&r%5B%5D=Region&r%5B%5D=Country+GLOBALLOBS&r%5B%5D=Order+Method&r%5B%5D=Cancel+Reason&r%5B%5D=Cancel+Reason+Type&r%5B%5D=Product+LOB&r%5B%5D=Product+Sub+Family&r%5B%5D=Product+Class&f%5B%5D=Sales+District+Category&f%5B%5D=Channel&f%5B%5D=Finance+Region&f%5B%5D=Order+Product+Type&salesDistCat%5B%5D=Sales&chnl%5B%5D=ONL&finRgn%5B%5D=Greater+China&ordProdType%5B%5D=STANDARD+ORDER&radio%5B%5D=a_v&altTax%5B%5D=a_v&bpItem%5B%5D=a_v&bpSubItem%5B%5D=a_v&cpnId%5B%5D=a_v&createTool%5B%5D=a_v&lnecreateTool%5B%5D=a_v&crosBrdr%5B%5D=a_v&csfID%5B%5D=a_v&custGrp%5B%5D=a_v&dsCt%5B%5D=a_v&emeaShip1%5B%5D=a_v&finInd%5B%5D=a_v&giftCd%5B%5D=a_v&giftEligInd%5B%5D=a_v&multiCrdInd%5B%5D=a_v&iCustNt%5B%5D=a_v&busAdd%5B%5D=a_v&busAddItm%5B%5D=a_v&iphLk%5B%5D=a_v&itmShpcnd%5B%5D=a_v&prtlDel%5B%5D=a_v&oneClick%5B%5D=a_v&payId%5B%5D=a_v&persInd%5B%5D=a_v&preAuth%5B%5D=a_v&qotInd%5B%5D=a_v&Requote_Ind%5B%5D=a_v&refurbInd%5B%5D=a_v&replOrder%5B%5D=a_v&retailStrId%5B%5D=a_v&runRate%5B%5D=a_v&salesQuote%5B%5D=a_v&slsOrg%5B%5D=a_v&shipCondCd%5B%5D=a_v&sapNte%5B%5D=a_v&tmsFlg%5B%5D=a_v&shipFlg%5B%5D=a_v&preFlg%5B%5D=a_v&sigRsn%5B%5D=a_v&shiPlnt%5B%5D=a_v&shipQuote%5B%5D=a_v&smbTierGrp%5B%5D=a_v&soldCust%5B%5D=a_v&sftID%5B%5D=a_v&thdPtypic%5B%5D=a_v&volOrdQty%5B%5D=a_v&vatflg%5B%5D=a_v&multiAdd%5B%5D=a_v&shipCompFlg%5B%5D=a_v&woHld%5B%5D=a_v

所以我的问题是如何使用python请求下载大数据文件?谢谢

0 个答案:

没有答案