Question

我正在尝试使用此问题中的代码将文件上传到S3存储桶：https://stackoverflow.com/a/15087468/291372。我正在使用boto2（boto3有太多的依赖项）。我尝试了很多方法，但没有人适合我。检查CORS的存储桶并设置为允许来自＆＃34; *＆＃34;

这是我的代码：

# -*- coding: utf-8 -*-

import boto
import boto.s3
import sys
from boto.s3.key import Key


AWS_ACCESS_KEY_ID = 'XXXXXXXXXXXXX'
AWS_SECRET_ACCESS_KEY = 'YYYYYYYYYYYYYYYYYYYYYy'
S3_BUCKET = 'ZZZZZZZZZZZZZZZZZZZZZZ'

conn = boto.connect_s3(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
bucket = conn.get_bucket(S3_BUCKET)

testfile = "test.jpg"
print('{}: Uploading {} to Amazon S3 bucket {}'.format(datetime.now().time().isoformat(), testfile, S3_BUCKET))

def percent_cb(complete, total):
    print(datetime.now().time().isoformat(), complete, total)
#    sys.stdout.write('.')
#    sys.stdout.flush()


k = Key(bucket)
k.key = 'my test file'
k.set_contents_from_filename(testfile, cb=percent_cb, num_cb=5)

这是我在控制台中可以看到的结果：

19:40:19.760703: Uploading test.jpg to Amazon S3 bucket pickettagent-phase2
19:40:21.394796 0 85937
19:40:22.061834 24576 85937
19:40:24.235959 0 85937
19:40:24.973001 24576 85937
19:40:27.542148 0 85937
19:40:30.110295 0 85937
19:40:38.648783 0 85937
19:40:49.520405 0 85937
19:41:12.959745 0 85937
19:41:13.644785 24576 85937
Traceback (most recent call last):
 ......
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

有时，第一列（complete）仅包含零。

该代码失败，异常为ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

我试图上传的文件大约是85Kb，所以它应该在一秒钟内上传，但实际上整个过程（从开始到异常）大约需要40-60秒。

我尝试了其他解决方案（tinys3，flask-s3，flask-upload），没有任何效果！

如何将文件上传到S3存储桶？也许我错过了某种权限？

Answer 1

试试这段代码，我用它来上传大量文件（其中大部分都是＆gt; 1MB）：

from boto.s3.connection import S3Connection
from contextlib import contextmanager

@contextmanager
def connect_to_s3():
    conn = S3Connection(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)

    try:
        yield conn
    finally:
        conn.close()

with connect_to_s3() as conn:
    bucket = conn.get_bucket(S3_BUCKET)
    key = bucket.new_key()
    key.set_contents_from_filename(file_name, cb=percent_cb, num_cb=5)

Boto2文件上传提供了ConnectionResetError

1 个答案: