Question

我正在尝试通过Boto 2.38.0和python 3.4.3连接Amazon S3。 S3帐户归另一家公司所有，他们只授予这些权限：

"Statement":
[
    {
        "Effect": "Allow",
        "Action": "s3:ListBucket",
        "Resource": "arn:axs:s3:::GA-Exports",
        "Condition":{
            "StringLike":
            {
                "s3.prefix": "Events_3112/*"
            }
        }
    },{
        "Effect": "Allow",
        "Action": 
        [
            "s3:GetObject",
            "s3.GetObjectAcl",
            "s3.GetBucketAcl"
        ],
        "Resource": "arn:axs:s3:::GA-Exports/Events_3112/*",
        "Condition": {}
    }
]

如果我设置名称，我可以连接并检索特定文件。但是我需要从S3检索所有数据（例如，通过脚本确定我尚未下载的文件）。

from boto.s3.connection import S3Connection
from boto.s3.connection import OrdinaryCallingFormat
s3_connection = S3Connection(access_key, secret_key,calling_format=OrdinaryCallingFormat())
bucket = s3_connection.get_bucket(__bucket_name, validate=False)
key = bucket.get_key(file_name)

有效，但

all_buckets = s3_connection.get_all_buckets()

引发错误

S3ResponseError: S3ResponseError: 403 Forbidden
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>19D20ADCFFC899ED</RequestId><HostId>eI4CzQqAvOnjcXJNZyyk+drFHjO9+yj0EtP+vJ5f/D7D4Dh2HFL3UvCacy9nP/wT</HostId></Error>

使用S3 Browser软件，我可以右键单击＆gt; “导出文件列表”，并得到我需要的东西。但是我怎么能在python中做到这一点？

编辑： 终于找到了答案：

bucket_name = 'GA-Exports'
s3_connection = S3Connection(access_key, secret_key, calling_format=OrdinaryCallingFormat())
bucket = s3_connection.get_bucket(bucket_name, validate=False)
for key in bucket.list(prefix='Events_3112/DEV/'):
    print(key.name, key.size, key.last_modified)

感谢您的帮助！：）

Answer 1

您将不被允许获得所有存储桶，权限表示您只能为“GA-Exports”列出存储桶内容：

from boto.s3.connection import S3Connection
from boto.s3.connection import OrdinaryCallingFormat
# this is to avoid a 301 mover permanently when used OrdinaryCallingFormat
if '.' in __bucket_name:
    conn = S3Connection(access_key, secret_key, calling_format=OrdinaryCallingFormat())
else:
    conn = S3Connection(access_key, secret_key)

bucket = conn.get_bucket(__bucket_name, validate=False)
l = bucket.list(prefix='Events_3112/') # now l is a list of objects within the bucket
# other option is to use bucket.get_all_keys()
for key in l:
    print l # or whatever you want to do with each file name
    # Recall this is only the filename not the file perse :-D

在http://boto.readthedocs.org/en/latest/ref/s3.html#module-boto.s3.bucket

中查看完整的存储桶对象引用

编辑：当通过ordinarycallingformat访问S3时收到301永久移动错误时添加了修复。添加@garnaat对前缀的评论也是（thx！）

AmazonS3 - 根据特定权限连接Python Boto

1 个答案: