Question

我正在尝试使用boto 下载S3存储桶中存在的日志文件。不使用s3cmd和其他工具背后的原因是我不希望我的代码依赖于某种软件/工具，以便其他人也可以直接使用我的代码而不必担心下载一些其他依赖项。

我得到以下堆栈跟踪。我看到了各种相关的帖子，但没有一个能解决我的问题。

Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/fabric/main.py", line 743, in main
*args, **kwargs
File "/Library/Python/2.7/site-packages/fabric/tasks.py", line 405, in execute
results['<local-only>'] = task.run(*args, **new_kwargs)
File "/Library/Python/2.7/site-packages/fabric/tasks.py", line 171, in run
return self.wrapped(*args, **kwargs)
File "/pgbadger/pgbadger_html.py", line 86, in dlogs
s3 = S3()
File "/pgbadger/pgbadger_html.py", line 46, in __init__
self.bucket = self._get_bucket(self.log_bucket)
File "/pgbadger/pgbadger_html.py", line 65, in _get_bucket
return self.s3_conn.get_bucket(bucket)
File "/Library/Python/2.7/site-packages/boto/s3/connection.py", line 471, in get_bucket
return self.head_bucket(bucket_name, headers=headers)
File "/Library/Python/2.7/site-packages/boto/s3/connection.py", line 518, in head_bucket
response.status, response.reason, body)
boto.exception.S3ResponseError: S3ResponseError: 400 Bad Request

我看过代码，但我不知道为什么会收到此错误。我的代码如下：

from fabric.api import task
from fabric.api import env

S3_LOG_BUCKET = BUCKET-NAME


class S3(object):
    s3_conn = None
    log_bucket = S3_LOG_BUCKET
    region = region
    bucket = None
    env.host_string = REGION-NAME

    def __init__(self):
        self._s3_connect()
        self.bucket = self._get_bucket(self.log_bucket)

    def _s3_connect(self):
        if not self.s3_conn:
            self.s3_conn = boto.s3.connect_to_region(
                self.region,
                aws_access_key_id=AWS_ACCESS_KEY_ID,
                aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
            )
        if not self.s3_conn:
            raise ValueError('Invalid Region Name: {}'.format(region))

    def download_s3_logs(self):
        for l in self.bucket.list():
            key_string = str(l.key)
            l.get_contents_to_filename("/tempLogFiles/" + key_string)
            print l.key

    def _get_bucket(self, bucket):
        return self.s3_conn.get_bucket(bucket)


@task
def dlogs():
    s3 = S3()
    s3.download_s3_logs()

Answer 1

解决问题：在S3-Log-Bucket中，我提到了特定于我的存储桶的整个路径。就像我有我的存储桶，其中有多个文件夹。所以我提到了它的整个路径，但BOTO并不期望这种情况发生。因此，我只需提及桶名，而不是提及整个路径。

以前我在做 - ＆gt;

log_bucket = Bucket/Inner Folder 1/Inner Folder 2/.../ which was wrong

正确的做法 - ＆gt;

log_bucket = Bucket

获取boto.exception.S3ResponseError：S3ResponseError：400 Bad Request异常

1 个答案: