Question

我经常将大量的bzipped sql文件上传到S3，并注意到它最近因此错误而崩溃。可能是什么导致了这个？它总是与崩溃的文件相同，但是我可以上传更大的文件而没有问题所以它似乎不是一个大小限制。

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    An unexpected error has occurred.
  Please report the following lines to:
  s3tools-general@lists.sourceforge.net
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Traceback (most recent call last):
  File "/usr/bin/s3cmd", line 1040, in <module>
    main()
  File "/usr/bin/s3cmd", line 1020, in main
    cmd_func(args)
  File "/usr/bin/s3cmd", line 188, in cmd_object_put
    response = s3.object_put_uri(real_filename, uri_final, extra_headers)
  File "/usr/lib/python2.5/site-packages/S3/S3.py", line 195, in object_put_uri
    return self.object_put(filename, uri.bucket(), uri.object(), extra_headers)
  File "/usr/lib/python2.5/site-packages/S3/S3.py", line 175, in object_put
    response = self.send_file(request, file)
  File "/usr/lib/python2.5/site-packages/S3/S3.py", line 384, in send_file
    debug("MD5 sums: computed=%s, received=%s" % (md5_computed, response["headers"]["etag"]))
KeyError: 'etag'

Answer 1

S3没有在其响应中返回ETag [entity tag]。无论是什么原因导致ETag在响应中丢失，您的s3cmd版本都不会期望ETag不存在并中止。

据我所知，这应该不再是s3cmd的最新版本0.9.9.91中的一个问题：

## S3.py ##
...
# S3 from time to time doesn't send ETag back in a response :-(
# Force re-upload here.
if not response['headers'].has_key('etag'):
    response['headers']['etag'] = ''
...
debug("MD5 sums: computed=%s, received=%s" % (md5_computed, response["headers"]["etag"]))
...

确保您使用的是s3cmd版本0.9.9.91。

s3cmd版本0.9.9.91将重试上传。如果此特定文件的后续重试也失败，请更改S3.py以打印出完整的S3响应，或者在Linux上使用数据包嗅探器[tcpdump，在Windows上使用Wireshark]拦截相同的完整S3响应。它可能包含有关导致此行为的其他提示。

Amazon S3cmd在某些大型上传文件中崩溃

1 个答案: