从Google Colab下载文件时出现错误104

Question

尝试下载保存为~100MB HDF5文件的keras模型文件。

from google.colab import files
files.download('models/mlp_mnist_l.h5')

但是我一直得到以下ConnectionResetError

----------------------------------------
Exception happened during processing of request from ('::ffff:127.0.0.1', 43270, 0, 0)
Traceback (most recent call last):
  File "/usr/lib/python3.6/socketserver.py", line 317, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/usr/lib/python3.6/socketserver.py", line 348, in process_request
    self.finish_request(request, client_address)
  File "/usr/lib/python3.6/socketserver.py", line 361, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python3.6/socketserver.py", line 696, in __init__
    self.handle()
  File "/usr/lib/python3.6/http/server.py", line 418, in handle
    self.handle_one_request()
  File "/usr/lib/python3.6/http/server.py", line 406, in handle_one_request
method()
  File "/usr/lib/python3.6/http/server.py", line 639, in do_GET
    self.copyfile(f, self.wfile)
  File "/usr/lib/python3.6/http/server.py", line 800, in copyfile
    shutil.copyfileobj(source, outputfile)
  File "/usr/lib/python3.6/shutil.py", line 82, in copyfileobj
fdst.write(buf)
  File "/usr/lib/python3.6/socketserver.py", line 775, in write
    self._sock.sendall(b)
ConnectionResetError: [Errno 104] Connection reset by peer

我是否需要重新启动虚拟机并重试，但是我会丢失经过培训的模型文件，还是有其他方式下载文件？

Answer 1

将其保存到谷歌驱动器使用Pydrive

# Install the PyDrive wrapper & import libraries.
# This only needs to be done once in a notebook.
!pip install -U -q PyDrive
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
from google.colab import auth
from oauth2client.client import GoogleCredentials

# Authenticate and create the PyDrive client.
# This only needs to be done once in a notebook.
auth.authenticate_user()
gauth = GoogleAuth()
gauth.credentials = GoogleCredentials.get_application_default()
drive = GoogleDrive(gauth)

# Create & upload a file.
uploaded = drive.CreateFile({'title': 'filename.csv'})
uploaded.SetContentFile('filename.csv')
uploaded.Upload()
print('Uploaded file with ID {}'.format(uploaded.get('id')))

Answer 2

您还可以使用Google合作实验室中的项目结构下载文件。

Step 1: Click on the chevron-right icon at the top left corner.
Step 2: Select the Files tab.
Step 3: Right click on the file you want to download and select the Download option.

以下是屏幕截图：

Answer 3

我通过在其他地方托管我的文件解决了这个问题，比如在github或（在我的情况下，700mb培训数据集在我的个人网站上保存为.tar）。然后使用python或bash脚本，将文件从该服务器下载/复制到colab文件。不重置计算机。

将以下内容放入Colab的代码块中：

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

try:
    from urllib.request import urlopen # python 3
except ImportError:
    from urllib2 import urlopen # python 2
import sys
import tarfile
import tempfile
import shutil

url = "http://site.to/file.tar"
with tempfile.TemporaryFile() as tmp:
    print("downloading", url)
    shutil.copyfileobj(urlopen(url), tmp)
    print("extracting")
    tmp.seek(0)
    tar = tarfile.open(fileobj=tmp)
    tar.extractall()
    tar.close()
    print("done")