使用Python在Linux上下载文件时出现间歇性错误。

时间:2018-06-20 22:15:15

标签: python linux download urllib

在使用Python 2.7.12将文件从云下载到硬盘时, 导致此调用的不可预测的间歇性错误:

File "/usr/lib/python2.7/ftplib.py", line 312, in makepasv
  host, port = parse227(self.sendcmd('PASV'))

我想尝试建立主动FTP连接,但是找不到在urllib中关闭被动FTP的方法。

该代码在我的家用台式机(Mac上)上可以正常工作。仅当代码中出现错误 在云中的Linux服务器上运行。实际上,该代码已有很长的历史,并且只是行为不当而已 今年。

我尝试了多种下载文件的方法。

方法1:此代码段是原始方法:

testfile = urllib.URLopener()
testfile.retrieve( url, self.pathstrng )

它会产生一个错误跟踪,结尾为:

Traceback (most recent call last):
  ...
  File "/home/code/storage3.py", line 452, in download_from_web_to_leafplace
    testfile.retrieve( url, self.pathstrng )
  File "/usr/lib/python2.7/urllib.py", line 245, in retrieve
    fp = self.open(url, data)
  File "/usr/lib/python2.7/urllib.py", line 213, in open
    return getattr(self, name)(url)
  File "/usr/lib/python2.7/urllib.py", line 558, in open_ftp
    (fp, retrlen) = self.ftpcache[key].retrfile(file, type)
  File "/usr/lib/python2.7/urllib.py", line 906, in retrfile
    conn, retrlen = self.ftp.ntransfercmd(cmd)
  File "/usr/lib/python2.7/ftplib.py", line 334, in ntransfercmd
    host, port = self.makepasv()
  File "/usr/lib/python2.7/ftplib.py", line 312, in makepasv
    host, port = parse227(self.sendcmd('PASV'))
  File "/usr/lib/python2.7/ftplib.py", line 830, in parse227
    raise error_reply, resp
IOError

方法2:今天,我尝试了这段代码片段

import wget
wget.download(url, self.pathstrng )

由于wget.py使用urllib,因此毫无疑问会产生以相同的结尾的错误:

Traceback (most recent call last):
  ...
  File "/home/code/storage3.py", line 467, in download_from_web_to_leafplace
    wget.download(url, self.pathstrng )
  File "/home/user4/.local/lib/python2.7/site-packages/wget.py", line 526, in download
    (tmpfile, headers) = ulib.urlretrieve(binurl, tmpfile, callback)
  File "/usr/lib/python2.7/urllib.py", line 98, in urlretrieve
    return opener.retrieve(url, filename, reporthook, data)
  File "/usr/lib/python2.7/urllib.py", line 245, in retrieve
    fp = self.open(url, data)
  File "/usr/lib/python2.7/urllib.py", line 213, in open
    return getattr(self, name)(url)
  File "/usr/lib/python2.7/urllib.py", line 558, in open_ftp
    (fp, retrlen) = self.ftpcache[key].retrfile(file, type)
  File "/usr/lib/python2.7/urllib.py", line 906, in retrfile
    conn, retrlen = self.ftp.ntransfercmd(cmd)
  File "/usr/lib/python2.7/ftplib.py", line 334, in ntransfercmd
    host, port = self.makepasv()
  File "/usr/lib/python2.7/ftplib.py", line 312, in makepasv
    host, port = parse227(self.sendcmd('PASV'))
  File "/usr/lib/python2.7/ftplib.py", line 830, in parse227
    raise error_reply, resp
IOError

方法3:我尝试过requests

            r = requests.get(url, stream=True)
            with open(self.pathstrng, 'wb') as f:
                for chunk in r.iter_content():
                    f.write(chunk)

并获得此追溯:

Traceback (most recent call last):
  ...
  File "/home/code/storage3.py", line 482, in download_from_web_to_leafplace
    r = requests.get(url, stream=True)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 55, in get
    return request('get', url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 335, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 433, in send
    adapter = self.get_adapter(url=request.url)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 474, in get_adapter
    raise InvalidSchema("No connection adapters were found for '%s'" % url)
requests.exceptions.InvalidSchema

如果诊断困难,我将对一致的解决方法感到满意。如前所述,该问题有时不会发生,因此我能够通过重新运行多次来最终下载所有文件(一次下载一个)。谢谢。

0 个答案:

没有答案