我正在尝试将最后一个文件上传到AmazonS3。似乎唯一的方法是获取所有密钥元数据,按日期排序,然后按名称加载正确的元数据。
conn = S3Connection(access_key, secret_key, calling_format=OrdinaryCallingFormat())
bucket = conn.get_bucket(bucket_name, validate=False)
for item in conn.make_request("HEAD", bucket_name).getheaders():
print('bucket header ->',item)
boto.set_stream_logger('paws')
for current_key in bucket.list():
print('in loop')
print(current_key)
#current_key = bucket.get_key(current_key.name)
#print(current_key.get_metadata('revision'))
以下是控制台结果:
bucket header -> ('x-amz-id-2', '25NUgCAlVd2PMfzIgEI0bzpvNBkXasFL9cpxpf3wUJbxnLBCuf0VLf1wiul13Z2lF06qNwfF3ss=')
bucket header -> ('x-amz-request-id', 'AF98E73E5BCF1838')
bucket header -> ('Date', 'Tue, 19 May 2015 08:21:29 GMT')
bucket header -> ('Last-Modified', 'Tue, 28 Oct 2014 11:30:14 GMT')
bucket header -> ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"')
bucket header -> ('Accept-Ranges', 'bytes')
bucket header -> ('Content-Type', 'application/octet-stream')
bucket header -> ('Content-Length', '0')
bucket header -> ('Server', 'AmazonS3')
2015-05-19 10:21:25,180 paws [DEBUG]:path=/GA-Exports/Events_3112/
2015-05-19 10:21:25,180 paws [DEBUG]:auth_path=/GA-Exports/Events_3112/
2015-05-19 10:21:25,180 paws [DEBUG]:Method: GET
2015-05-19 10:21:25,180 paws [DEBUG]:Path: /GA-Exports/Events_3112/
2015-05-19 10:21:25,180 paws [DEBUG]:Data:
2015-05-19 10:21:25,180 paws [DEBUG]:Headers: {}
2015-05-19 10:21:25,183 paws [DEBUG]:Host: s3.amazonaws.com:443
2015-05-19 10:21:25,183 paws [DEBUG]:Port: 443
2015-05-19 10:21:25,183 paws [DEBUG]:Params: {}
2015-05-19 10:21:25,183 paws [DEBUG]:establishing HTTPS connection: host=s3.amazonaws.com, kwargs={'timeout': 70, 'port': 443}
2015-05-19 10:21:25,200 paws [DEBUG]:Token: None
2015-05-19 10:21:25,200 paws [DEBUG]:StringToSign:
GET
Tue, 19 May 2015 08:21:25 GMT
/GA-Exports/Events_3112/
2015-05-19 10:21:25,200 paws [DEBUG]:Signature:
AWS [_accesskey_]:oIo0Qk6pCn+pMPzarqRs4Pb3AvE=
2015-05-19 10:21:25,200 paws [DEBUG]:Final headers: {'Date': 'Tue, 19 May 2015 08:21:25 GMT', 'User-Agent': 'Boto/2.38.0 Python/3.4.3 Windows/8', 'Authorization': 'AWS [_accesskey_]:oIo0Qk6pCn+pMPzarqRs4Pb3AvE=', 'Content-Length': '0'}
2015-05-19 10:21:25,669 paws [DEBUG]:Response headers: [('x-amz-id-2', 'FzO+CIljQO4DKnEusv0Lm/qns5C1hP+REZ0YlYp+TV+XlxT6AKLacHG//UWMOsKGz1vuuVIZ2t4='), ('x-amz-request-id', '52D92D3D21FFD93D'), ('Date', 'Tue, 19 May 2015 08:21:30 GMT'), ('Last-Modified', 'Tue, 28 Oct 2014 11:30:14 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/octet-stream'), ('Content-Length', '0'), ('Server', 'AmazonS3')]
2015-05-19 10:21:25,669 paws [DEBUG]:b''
Traceback (most recent call last):
File "D:\Python\lib\xml\sax\expatreader.py", line 207, in feed
self._parser.Parse(data, isFinal)
xml.parsers.expat.ExpatError: no element found: line 1, column 0
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Francois\OneDrive\IDE\Workspace\eclipse\Python_test\etltest.py", line 33, in <module>
for current_key in bucket.list():
File "D:\Python\lib\site-packages\boto\s3\bucketlistresultset.py", line 34, in bucket_lister
encoding_type=encoding_type)
File "D:\Python\lib\site-packages\boto\s3\bucket.py", line 472, in get_all_keys
'', headers, **params)
File "D:\Python\lib\site-packages\boto\s3\bucket.py", line 406, in _get_all
xml.sax.parseString(body, h)
File "D:\Python\lib\xml\sax\__init__.py", line 46, in parseString
parser.parse(inpsrc)
File "D:\Python\lib\xml\sax\expatreader.py", line 107, in parse
xmlreader.IncrementalParser.parse(self, source)
File "D:\Python\lib\xml\sax\xmlreader.py", line 125, in parse
self.close()
File "D:\Python\lib\xml\sax\expatreader.py", line 217, in close
self.feed("", isFinal = 1)
File "D:\Python\lib\xml\sax\expatreader.py", line 211, in feed
self._err_handler.fatalError(exc)
File "D:\Python\lib\xml\sax\handler.py", line 38, in fatalError
raise exception
xml.sax._exceptions.SAXParseException: <unknown>:1:0: no element found
bucket.get_all_keys()也会引发相同的异常......
我是否有权按名称获取特定文件但不能列出存储桶密钥的权限?
答案 0 :(得分:0)
您的帐户中的运行代码已成功运行,因此很可能是与权限相关的问题。
需要注意的重要一点是,列出存储桶的权限适用于存储桶,而存储桶中对象的权限适用于存储桶的内容。< / p>
您需要一个应用此类权限的政策(取自Example )Policies for Administering AWS Resources): 请注意, 最简单的测试是使用允许所有S3功能的策略尝试您的代码,然后缩小策略以满足要求。{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:ListAllMyBuckets",
"Resource": "arn:aws:s3:::*"
},
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": "arn:aws:s3:::EXAMPLE-BUCKET-NAME"
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": "arn:aws:s3:::EXAMPLE-BUCKET-NAME/*"
}
]
}
ListBucket
使用"arn:aws:s3:::EXAMPLE-BUCKET-NAME"
,而PutObject
使用"arn:aws:s3:::EXAMPLE-BUCKET-NAME/*"
。