Python boto bucket.list()返回“找不到元素”

时间:2015-05-19 08:41:12

标签: python amazon-s3 boto

我正在尝试将最后一个文件上传到AmazonS3。似乎唯一的方法是获取所有密钥元数据,按日期排序,然后按名称加载正确的元数据。

conn = S3Connection(access_key, secret_key, calling_format=OrdinaryCallingFormat())

bucket = conn.get_bucket(bucket_name, validate=False)

for item in conn.make_request("HEAD", bucket_name).getheaders():
    print('bucket header ->',item)

boto.set_stream_logger('paws')

for current_key in bucket.list():
    print('in loop')
    print(current_key)
    #current_key = bucket.get_key(current_key.name)
    #print(current_key.get_metadata('revision'))

以下是控制台结果:

bucket header -> ('x-amz-id-2', '25NUgCAlVd2PMfzIgEI0bzpvNBkXasFL9cpxpf3wUJbxnLBCuf0VLf1wiul13Z2lF06qNwfF3ss=')
bucket header -> ('x-amz-request-id', 'AF98E73E5BCF1838')
bucket header -> ('Date', 'Tue, 19 May 2015 08:21:29 GMT')
bucket header -> ('Last-Modified', 'Tue, 28 Oct 2014 11:30:14 GMT')
bucket header -> ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"')
bucket header -> ('Accept-Ranges', 'bytes')
bucket header -> ('Content-Type', 'application/octet-stream')
bucket header -> ('Content-Length', '0')
bucket header -> ('Server', 'AmazonS3')
2015-05-19 10:21:25,180 paws [DEBUG]:path=/GA-Exports/Events_3112/
2015-05-19 10:21:25,180 paws [DEBUG]:auth_path=/GA-Exports/Events_3112/
2015-05-19 10:21:25,180 paws [DEBUG]:Method: GET
2015-05-19 10:21:25,180 paws [DEBUG]:Path: /GA-Exports/Events_3112/
2015-05-19 10:21:25,180 paws [DEBUG]:Data: 
2015-05-19 10:21:25,180 paws [DEBUG]:Headers: {}
2015-05-19 10:21:25,183 paws [DEBUG]:Host: s3.amazonaws.com:443
2015-05-19 10:21:25,183 paws [DEBUG]:Port: 443
2015-05-19 10:21:25,183 paws [DEBUG]:Params: {}
2015-05-19 10:21:25,183 paws [DEBUG]:establishing HTTPS connection: host=s3.amazonaws.com, kwargs={'timeout': 70, 'port': 443}
2015-05-19 10:21:25,200 paws [DEBUG]:Token: None
2015-05-19 10:21:25,200 paws [DEBUG]:StringToSign:
GET


Tue, 19 May 2015 08:21:25 GMT
/GA-Exports/Events_3112/
2015-05-19 10:21:25,200 paws [DEBUG]:Signature:
AWS [_accesskey_]:oIo0Qk6pCn+pMPzarqRs4Pb3AvE=
2015-05-19 10:21:25,200 paws [DEBUG]:Final headers: {'Date': 'Tue, 19 May 2015 08:21:25 GMT', 'User-Agent': 'Boto/2.38.0 Python/3.4.3 Windows/8', 'Authorization': 'AWS [_accesskey_]:oIo0Qk6pCn+pMPzarqRs4Pb3AvE=', 'Content-Length': '0'}
2015-05-19 10:21:25,669 paws [DEBUG]:Response headers: [('x-amz-id-2', 'FzO+CIljQO4DKnEusv0Lm/qns5C1hP+REZ0YlYp+TV+XlxT6AKLacHG//UWMOsKGz1vuuVIZ2t4='), ('x-amz-request-id', '52D92D3D21FFD93D'), ('Date', 'Tue, 19 May 2015 08:21:30 GMT'), ('Last-Modified', 'Tue, 28 Oct 2014 11:30:14 GMT'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/octet-stream'), ('Content-Length', '0'), ('Server', 'AmazonS3')]
2015-05-19 10:21:25,669 paws [DEBUG]:b''
Traceback (most recent call last):
  File "D:\Python\lib\xml\sax\expatreader.py", line 207, in feed
    self._parser.Parse(data, isFinal)
xml.parsers.expat.ExpatError: no element found: line 1, column 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Francois\OneDrive\IDE\Workspace\eclipse\Python_test\etltest.py", line 33, in <module>
    for current_key in bucket.list():
  File "D:\Python\lib\site-packages\boto\s3\bucketlistresultset.py", line 34, in bucket_lister
    encoding_type=encoding_type)
  File "D:\Python\lib\site-packages\boto\s3\bucket.py", line 472, in get_all_keys
    '', headers, **params)
  File "D:\Python\lib\site-packages\boto\s3\bucket.py", line 406, in _get_all
    xml.sax.parseString(body, h)
  File "D:\Python\lib\xml\sax\__init__.py", line 46, in parseString
    parser.parse(inpsrc)
  File "D:\Python\lib\xml\sax\expatreader.py", line 107, in parse
    xmlreader.IncrementalParser.parse(self, source)
  File "D:\Python\lib\xml\sax\xmlreader.py", line 125, in parse
    self.close()
  File "D:\Python\lib\xml\sax\expatreader.py", line 217, in close
    self.feed("", isFinal = 1)
  File "D:\Python\lib\xml\sax\expatreader.py", line 211, in feed
    self._err_handler.fatalError(exc)
  File "D:\Python\lib\xml\sax\handler.py", line 38, in fatalError
    raise exception
xml.sax._exceptions.SAXParseException: <unknown>:1:0: no element found

bucket.get_all_keys()也会引发相同的异常......

我是否有权按名称获取特定文件但不能列出存储桶密钥的权限?

1 个答案:

答案 0 :(得分:0)

您的帐户中的运行代码已成功运行,因此很可能是与权限相关的问题。

需要注意的重要一点是,列出存储桶的权限适用于存储桶,而存储桶中对象的权限适用于存储桶的内容。< / p>

您需要一个应用此类权限的政策(取自Example )Policies for Administering AWS Resources):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:ListAllMyBuckets",
      "Resource": "arn:aws:s3:::*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:GetBucketLocation"
      ],
      "Resource": "arn:aws:s3:::EXAMPLE-BUCKET-NAME"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject"
      ],
      "Resource": "arn:aws:s3:::EXAMPLE-BUCKET-NAME/*"
    }
  ]
}

请注意,ListBucket使用"arn:aws:s3:::EXAMPLE-BUCKET-NAME",而PutObject使用"arn:aws:s3:::EXAMPLE-BUCKET-NAME/*"

最简单的测试是使用允许所有S3功能的策略尝试您的代码,然后缩小策略以满足要求。