我尝试使用python下载视频。
我尝试使用多线程加速下载,但我无法在requests
或urllib2
中提出解决方案。
此外,如果任何人都可以提供有关如何解决它的代码,那将非常有用。
以下是我尝试的代码:
import requests
http_proxy = "http://edcguest:edcguest@172.31.100.29:3128"
https_proxy = "https://edcguest:edcguest@172.31.100.29:3128"
ftp_proxy = "ftp://edcguest:edcguest@172.31.100.29:3128"
proxyDict = {
"http" : http_proxy,
"https" : https_proxy,
"ftp" : ftp_proxy
}
def download_file(url):
resume_byte_pos = 0
end_byte_pos = 432526330
# NOTE the stream=True parameter
resume_header = {'Range': 'bytes=%d-%d(resume_byte_pos,end_byte_pos)}
#r = requests.get(url, stream=True, proxies=proxyDict)
r = requests.get(url, stream=True,proxies=proxyDict,headers=resume_header)
print r.headers
with open('ab2.mp4', 'wb') as f:
for chunk in r.iter_content(chunk_size=1024):
if chunk: # filter out keep-alive new chunks
f.write(chunk)
f.flush()
download_file('https://r2---sn-o3o-qxal.googlevideo.com/videoplayback?key=yt5&sver=3&signature=3D4D50B11C6206B737185B7A9887A72FE356C6DF.87458BB3BF357CEF131BEDF0C0ED3DC08F087646&upn=8n6wa_1gM_o&source=youtube&requiressl=yes&mime=video%2Fmp4&ip=14.139.249.194&expire=1442331973&ratebypass=yes&lmt=1441845538878516&mm=31&ipbits=0&mn=sn-o3o-qxal&pl=24&sparams=dur%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&fexp=9408710%2C9409069%2C9409170%2C9412773%2C9415365%2C9415485%2C9415942%2C9416023%2C9416126%2C9416333%2C9416729%2C9417707%2C9417710%2C9417818%2C9418153%2C9418162%2C9418200%2C9418245%2C9418448%2C9418986%2C9419773%2C9419788%2C9419837%2C9420348%2C9420777%2C9420798&id=o-ALtyLWP7o7PqDhINh6FWp4v4FC8-3pQoZ0UH4COW6v5p&mt=1442310331&dur=8384.609&mv=m&initcwndbps=4191250&ms=au&itag=18&cpn=zzjuCmNROtaupQMW&ptk=Apple%252Bvid&oid=ffsQQyXI443h2PgMzMjp-g&ptchn=E_M8A5yxnLfW0KghEeajjw&pltype=content&c=WEB&cver=html5')
答案 0 :(得分:1)
urllib2
is not thread-safe,如果与threading
一起使用,则适用警告。
但是,urllib3
确实是 thread safe,因此您可以考虑使用它。
或者,如果你有一个基于urllib2
的库,你不想重写你将用于批量下载,你可以探索任务队列,如{ {3}}或Celery。