我正在运行一个Amazon EC2(ubuntu)实例,该实例每天输出一个JSON文件。我现在正尝试将此JSON复制到Amazon S3,以便最终将其下载到我的本地计算机上。按照此处(reading in a file from ubuntu (AWS EC2) on local machine?)的说明进行操作,我正在使用boto3将JSON从ubuntu复制到S3:
import boto3
print("This script uploads the SF events JSON to s3")
ACCESS_ID = 'xxxxxxxx'
ACCESS_KEY = 'xxxxxxx'
s3 = boto3.resource('s3',
aws_access_key_id=ACCESS_ID,
aws_secret_access_key= ACCESS_KEY)
def upload_file_to_s3(s3_path, local_path):
bucket = s3_path.split('/')[2]
print(bucket)
file_path = '/'.join(s3_path.split('/')[3:])
print(file_path)
response = s3.Object(bucket, file_path).upload_file(local_path)
print(response)
s3_path = "s3://mybucket/sf_events.json"
local_path = "/home/ubuntu/bandsintown/sf_events.json"
upload_file_to_s3(s3_path, local_path)
我在这里使用的凭证来自在Amazon Identity and Access Management(IAM)中创建一个新用户:随附的屏幕截图。
但是,当我运行此脚本时,出现以下错误:
boto3.exceptions.S3UploadFailedError: Failed to upload /home/ubuntu/bandsintown/sf_events.json to mybucket/sf_events.json: An error occurred (AccessDenied) when calling the PutObject operation: Access Denied
我还尝试将IAM角色附加到EC2实例,并赋予该角色完全s3权限-但仍然没有运气(请参见下图)。
这似乎是权限问题-谁能告诉我如何开始解决此问题?我需要Amazon CLI吗?我还在boto3文档中读到我的脚本中可能需要aws_session_token参数。
很简单,我迷路了。谢谢。
答案 0 :(得分:1)
由于它是ec2,因此可以为实例分配IAM角色,并为角色分配权限。另外,您无需在代码中对凭据进行硬编码。
https://aws.amazon.com/premiumsupport/knowledge-center/assign-iam-role-ec2-instance/
您可以将此政策用于S3上传
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "statement1",
"Effect": "Allow",
"Action": ["s3:PutObject","s3:PutObjectAcl"],
"Resource": "arn:aws:s3:::examplebucket/*"
}
]
}
以下是您将策略附加到IAM角色的方法: https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_manage-attach-detach.html
并将代码更改为:
s3 = boto3.resource('s3')