Question

我有一个s3存储桶，其中包含一堆文件，这些文件我想从我的lambda（同一个帐户创建的lambda和s3存储桶）中访问：

def list_all():
  s3 = boto3.client('s3')
  bucket = 'my-bucket'
  resp = s3.list_objects(Bucket=bucket, MaxKeys=10)
  print("s3.list_objects returns", resp)

这会产生如下错误：

{
  "errorMessage": "An error occurred (AccessDenied) when calling the 
                   ListObjects operation: Access Denied",
  "errorType": "ClientError",
  "stackTrace": [
  [
    "/var/task/lambda_function.py",
    41,
    "lambda_handler",
    "list_all()"
  ], ...

我的存储桶设置在aws上显示如下：

{
"Version": "2012-10-17",
"Statement": [
    {
        "Sid": "AddPerm",
        "Effect": "Allow",
        "Principal": "*",
        "Action": "s3:GetObject",
        "Resource": "arn:aws:s3:::my-bucket/*"
    }
  ]
}

我有两个问题：

1）我应该将“操作”字段设置为什么，以便可以使用boto3列出lambda中任何文件夹中的所有文件？

2）我应该将本金设置为什么，以便只有我的aws帐户（例如，当我运行lambda时）才能访问存储桶？

Answer 1

Lambda失败的原因是使用df = df.pivot_table(index=['indices'], columns=['column'], values=['start_value','end_value','delta','name','unit'], aggfunc=lambda x: x.sum(min_count=1) ) print (df) end_value name unit column '1nan' '1nan' 'other' '1nan' 'other' indices A 1000.0 'test' 'test2' 'USD' 'USD'，您的 lambda函数需要具有针对单个存储桶（无对象）的IAM权限listObjects需要通配符）(docs)。

即您应该set your lambda's IAM policy执行以下操作：

s3:ListBucket

Answer 2

def list_s3_by_prefix(bucket, key_prefix, filter_func=None):
    next_token = ''
    all_keys = []
    while True:
        if next_token:
            res = s3.list_objects_v2(
                Bucket=bucket,
                ContinuationToken=next_token,
                Prefix=key_prefix)
        else:
            res = s3.list_objects_v2(
                Bucket=bucket,
                Prefix=key_prefix)

        if 'Contents' not in res:
            break

        if res['IsTruncated']:
            next_token = res['NextContinuationToken']
        else:
            next_token = ''

        if filter_func:
            keys = ["s3://{}/{}".format(bucket, item['Key']) for item in res['Contents'] if filter_func(item['Key'])]
        else:
            keys = ["s3://{}/{}".format(bucket, item['Key']) for item in res['Contents']]

        all_keys.extend(keys)

        if not next_token:
            break
    print("find {} files in {}".format(len(all_keys), key_prefix))

使用boto3

2 个答案: