我正在尝试修改this AWS提供的CDK示例，以改用现有的存储桶。附加的documentation表示支持导入现有资源。到目前为止，我无法使用CDK向现有存储桶添加事件通知。
这是示例的修改后的版本：
class S3TriggerStack(core.Stack):
def __init__(self, scope: core.Construct, id: str, **kwargs) -> None:
super().__init__(scope, id, **kwargs)
# create lambda function
function = _lambda.Function(self, "lambda_function",
runtime=_lambda.Runtime.PYTHON_3_7,
handler="lambda-handler.main",
code=_lambda.Code.asset("./lambda"))
# **MODIFIED TO GET EXISTING BUCKET**
#s3 = _s3.Bucket(self, "s3bucket")
s3 = _s3.Bucket.from_bucket_arn(self, 's3_bucket',
bucket_arn='arn:<my_region>:::<my_bucket>')
# create s3 notification for lambda function
notification = aws_s3_notifications.LambdaDestination(function)
# assign notification for the s3 event type (ex: OBJECT_CREATED)
s3.add_event_notification(_s3.EventType.OBJECT_CREATED, notification)
尝试add_event_notification时会导致以下错误：
AttributeError: '_IBucketProxy' object has no attribute 'add_event_notification'
from_bucket_arn函数返回一个IBucket，而add_event_notification函数是Bucket类的方法，但是我似乎找不到其他方法做这个。也许不支持。任何帮助将不胜感激。
答案 0 :(得分：4)
对不起，由于声誉低下，我无法对上述出色的答案发表评论。但是我把James Irwin's answer做成了一个建构子。
关于“拒绝访问”的评论也花了我一些时间，但问题的关键是该功能为S3:putBucketNotificationConfiguration，但允许的IAM策略操作为S3:PutBucketNotification。 / p>
import * as cr from '@aws-cdk/custom-resources';
import * as logs from '@aws-cdk/aws-logs';
import * as s3 from '@aws-cdk/aws-s3';
import * as sqs from '@aws-cdk/aws-sqs';
import * as iam from '@aws-cdk/aws-iam';
import {Construct} from '@aws-cdk/core';
// You can drop this construct anywhere, and in your stack, invoke it like this:
// const s3ToSQSNotification = new S3NotificationToSQSCustomResource(this, 's3ToSQSNotification', existingBucket, queue);
export class S3NotificationToSQSCustomResource extends Construct {
constructor(scope: Construct, id: string, bucket: s3.IBucket, queue: sqs.Queue) {
super(scope, id);
// https://stackoverflow.com/questions/58087772/aws-cdk-how-to-add-an-event-notification-to-an-existing-s3-bucket
const notificationResource = new cr.AwsCustomResource(scope, id+"CustomResource", {
onCreate: {
service: 'S3',
action: 'putBucketNotificationConfiguration',
parameters: {
// This bucket must be in the same region you are deploying to
Bucket: bucket.bucketName,
NotificationConfiguration: {
QueueConfigurations: [
{
Events: ['s3:ObjectCreated:*'],
QueueArn: queue.queueArn,
}
]
}
},
physicalResourceId: <cr.PhysicalResourceId>(id + Date.now().toString()),
},
onDelete: {
service: 'S3',
action: 'putBucketNotificationConfiguration',
parameters: {
// This bucket must be in the same region you are deploying to
Bucket: bucket.bucketName,
// deleting a notification configuration involves setting it to empty.
NotificationConfiguration: {
}
},
physicalResourceId: <cr.PhysicalResourceId>(id + Date.now().toString()),
},
policy: cr.AwsCustomResourcePolicy.fromStatements([new iam.PolicyStatement({
// The actual function is PutBucketNotificationConfiguration.
// The "Action" for IAM policies is PutBucketNotification.
// https://docs.aws.amazon.com/AmazonS3/latest/dev/list_amazons3.html#amazons3-actions-as-permissions
actions: ["S3:PutBucketNotification"],
// allow this custom resource to modify this bucket
resources: [bucket.bucketArn],
})]),
logRetention: logs.RetentionDays.ONE_DAY,
});
// allow S3 to send notifications to our queue
// https://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html#grant-destinations-permissions-to-s3
queue.addToResourcePolicy(new iam.PolicyStatement({
principals: [new iam.ServicePrincipal("s3.amazonaws.com")],
actions: ["SQS:SendMessage"],
resources: [queue.queueArn],
conditions: {
ArnEquals: {"aws:SourceArn": bucket.bucketArn}
}
}));
// don't create the notification custom-resource until after both the bucket and queue
// are fully created and policies applied.
notificationResource.node.addDependency(bucket);
notificationResource.node.addDependency(queue);
}
}
答案 1 :(得分：2)
这是用于向包含过滤器的现有存储桶添加/替换lambda触发器的python解决方案。 @James Irwin的示例非常有帮助。
感谢@JørgenFrøland指出自定义资源配置将取代所有基于boto3文档https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.BucketNotification.put
请注意，他拒绝访问的问题是
because if you do putBucketNotificationConfiguration action the policy creates a s3:PutBucketNotificationConfiguration action but that action doesn't exist https://github.com/aws/aws-cdk/issues/3318#issuecomment-584737465
如果您使用AwsCustomResourcePolicy.fromSdkCalls设置策略，则会发生相同的问题
我添加了一个可能需要进一步限制的自定义策略。
s3_bucket = s3.Bucket.from_bucket_name(
self, 's3-bucket-by-name', 'existing-bucket-name')
trigger_lambda = _lambda.Function(
self,
'{id}-s3-trigger-lambda',
environment=lambda_env,
code=_lambda.Code.from_asset('./ladle-sink/'),
runtime=_lambda.Runtime.PYTHON_3_7,
handler='lambda_function.lambda_handler',
memory_size=512,
timeout=core.Duration.minutes(3))
trigger_lambda.add_permission(
's3-trigger-lambda-s3-invoke-function',
principal=iam.ServicePrincipal('s3.amazonaws.com'),
action='lambda:InvokeFunction',
source_arn=base_resources.incoming_documents_bucket.bucket_arn)
custom_s3_resource = _custom_resources.AwsCustomResource(
self,
's3-incoming-documents-notification-resource',
policy=_custom_resources.AwsCustomResourcePolicy.from_statements([
iam.PolicyStatement(
effect=iam.Effect.ALLOW,
resources=['*'],
actions=['s3:PutBucketNotification']
)
]),
on_create=_custom_resources.AwsSdkCall(
service="S3",
action="putBucketNotificationConfiguration",
parameters={
"Bucket": s3_bucket.bucket_name,
"NotificationConfiguration": {
"LambdaFunctionConfigurations": [
{
"Events": ['s3:ObjectCreated:*'],
"LambdaFunctionArn": trigger_lambda.function_arn,
"Filter": {
"Key": {
"FilterRules": [
{'Name': 'suffix', 'Value': 'html'}]
}
}
}
]
}
},
physical_resource_id=_custom_resources.PhysicalResourceId.of(
f's3-notification-resource-{str(uuid.uuid1())}'),
region=env.region
))
custom_s3_resource.node.add_dependency(
trigger_lambda.permissions_node.find_child(
's3-trigger-lambda-s3-invoke-function'))
答案 2 :(得分：2)
更新：原始答案中的源代码将覆盖存储桶的现有通知列表，这将导致无法添加新的lambda触发器。这是使用事件源处理提到的问题的解决方案。
Game development with unity原始： 我在TypeScript中使用了ubi的解决方案，并将其成功转换为Python。他的解决方案对我有用。
import aws_cdk {
aws_s3 as s3,
aws_cdk.aws_lambda as lambda_
aws_lambda_event_sources as event_src
}
import path as path
class S3LambdaTrigger(core.Stack):
def __init__(self, scope: core.Construct, id: str):
super().__init__(scope, id)
bucket = s3.Bucket(
self, "S3Bucket",
block_public_access=s3.BlockPublicAccess.BLOCK_ALL,
bucket_name='BucketName',
encryption=s3.BucketEncryption.S3_MANAGED,
versioned=True
)
fn = lambda_.Function(
self, "LambdaFunction",
runtime=lambda_.Runtime.NODEJS_10_X,
handler="index.handler",
code=lambda_.Code.from_asset(path.join(__dirname, "lambda-handler"))
)
fn.add_permission(
's3-service-principal',
principal=aws_iam.ServicePrincipal('s3.amazonaws.com')
)
fn.add_event_source(
event_src.S3EventSource(
bucket,
events=[s3.EventType.OBJECT_CREATED, s3.EventType.OBJECT_REMOVED],
filters=[s3.NotificationKeyFilter(prefix="subdir/", suffix=".txt")]
)
)
#!/usr/bin/env python
from typing import List
from aws_cdk import (
core,
custom_resources as cr,
aws_lambda as lambda_,
aws_s3 as s3,
aws_iam as iam,
)
class S3NotificationLambdaProps:
def __init__(self, bucket: s3.Bucket, function: lambda_.Function, events: List[str], prefix: str):
self.bucket = bucket
self.function = function
self.events = events
self.prefix = prefix
class S3NotificationLambda(core.Construct):
def __init__(self, scope: core.Construct, id: str, props: S3NotificationLambdaProps):
super().__init__(scope, id)
self.notificationResource = cr.AwsCustomResource(
self, f'CustomResource{id}',
on_create=cr.AwsSdkCall(
service="S3",
action="S3:putBucketNotificationConfiguration",
# Always update physical ID so function gets executed
physical_resource_id=cr.PhysicalResourceId.of(f'S3NotifCustomResource{id}'),
parameters={
"Bucket": props.bucket.bucket_name,
"NotificationConfiguration": {
"LambdaFunctionConfigurations": [{
"Events": props.events,
"LambdaFunctionArn": props.function.function_arn,
"Filter": {
"Key": {"FilterRules": [{"Name": "prefix", "Value": props.prefix}]}
}}
]
}
}
),
on_delete=cr.AwsSdkCall(
service="S3",
action="S3:putBucketNotificationConfiguration",
# Always update physical ID so function gets executed
physical_resource_id=cr.PhysicalResourceId.of(f'S3NotifCustomResource{id}'),
parameters={
"Bucket": props.bucket.bucket_name,
"NotificationConfiguration": {},
}
),
policy=cr.AwsCustomResourcePolicy.from_statements(
statements=[
iam.PolicyStatement(
actions=["S3:PutBucketNotification", "S3:GetBucketNotification"],
resources=[props.bucket.bucket_arn]
),
]
)
)
props.function.add_permission(
"AllowS3Invocation",
action="lambda:InvokeFunction",
principal=iam.ServicePrincipal("s3.amazonaws.com"),
source_arn=props.bucket.bucket_arn,
)
# don't create the notification custom-resource until after both the bucket and lambda
# are fully created and policies applied.
self.notificationResource.node.add_dependency(props.bucket)
self.notificationResource.node.add_dependency(props.function)
答案 3 :(得分：1)
我设法通过custom resource来解决这个问题。它是TypeScript，但应轻松转换为Python：
const uploadBucket = s3.Bucket.fromBucketName(this, 'BucketByName', 'existing-bucket');
const fn = new lambda.Function(this, 'MyFunction', {
runtime: lambda.Runtime.NODEJS_10_X,
handler: 'index.handler',
code: lambda.Code.fromAsset(path.join(__dirname, 'lambda-handler'))
});
const rsrc = new AwsCustomResource(this, 'S3NotificationResource', {
onCreate: {
service: 'S3',
action: 'putBucketNotificationConfiguration',
parameters: {
// This bucket must be in the same region you are deploying to
Bucket: uploadBucket.bucketName,
NotificationConfiguration: {
LambdaFunctionConfigurations: [
{
Events: ['s3:ObjectCreated:*'],
LambdaFunctionArn: fn.functionArn,
Filter: {
Key: {
FilterRules: [{ Name: 'suffix', Value: 'csv' }]
}
}
}
]
}
},
// Always update physical ID so function gets executed
physicalResourceId: 'S3NotifCustomResource' + Date.now().toString()
}
});
fn.addPermission('AllowS3Invocation', {
action: 'lambda:InvokeFunction',
principal: new iam.ServicePrincipal('s3.amazonaws.com'),
sourceArn: uploadBucket.bucketArn
});
rsrc.node.addDependency(fn.permissionsNode.findChild('AllowS3Invocation'));
这基本上是this example中列出的CloudFormation模板的CDK版本。有关可能的NotificationConfiguration参数，请参见AWS SDK上的文档。
答案 4 :(得分：1)
基于@ubi的答案
如果您不需要SingletonFunction，但需要Function +一些清理
像这样打电话：
const s3NotificationLambdaProps = < S3NotificationLambdaProps > {
bucket: bucket,
lambda: lambda,
events: ['s3:ObjectCreated:*'],
prefix: '', // or put some prefix
};
const s3NotificationLambda = new S3NotificationLambda(this, `${envNameUpperCase}S3ToLambdaNotification`, s3NotificationLambdaProps);
，构造将如下所示：
import * as cr from "@aws-cdk/custom-resources";
import * as s3 from "@aws-cdk/aws-s3";
import * as iam from "@aws-cdk/aws-iam";
import { Construct } from "@aws-cdk/core";
import * as lambda from "@aws-cdk/aws-lambda";
export interface S3NotificationLambdaProps {
bucket: s3.IBucket;
lambda: lambda.Function;
events: string[];
prefix: string;
}
export class S3NotificationLambda extends Construct {
constructor(scope: Construct, id: string, props: S3NotificationLambdaProps) {
super(scope, id);
const notificationResource = new cr.AwsCustomResource(
scope,
id + "CustomResource", {
onCreate: {
service: "S3",
action: "putBucketNotificationConfiguration",
parameters: {
// This bucket must be in the same region you are deploying to
Bucket: props.bucket.bucketName,
NotificationConfiguration: {
LambdaFunctionConfigurations: [{
Events: props.events,
LambdaFunctionArn: props.lambda.functionArn,
Filter: {
Key: {
FilterRules: [{
Name: "prefix",
Value: props.prefix
}],
},
},
}, ],
},
},
physicalResourceId: < cr.PhysicalResourceId > (
(id + Date.now().toString())
),
},
onDelete: {
service: "S3",
action: "putBucketNotificationConfiguration",
parameters: {
// This bucket must be in the same region you are deploying to
Bucket: props.bucket.bucketName,
// deleting a notification configuration involves setting it to empty.
NotificationConfiguration: {},
},
physicalResourceId: < cr.PhysicalResourceId > (
(id + Date.now().toString())
),
},
policy: cr.AwsCustomResourcePolicy.fromStatements([
new iam.PolicyStatement({
// The actual function is PutBucketNotificationConfiguration.
// The "Action" for IAM policies is PutBucketNotification.
// https://docs.aws.amazon.com/AmazonS3/latest/dev/list_amazons3.html#amazons3-actions-as-permissions
actions: ["S3:PutBucketNotification", "S3:GetBucketNotification"],
// allow this custom resource to modify this bucket
resources: [props.bucket.bucketArn],
}),
]),
}
);
props.lambda.addPermission("AllowS3Invocation", {
action: "lambda:InvokeFunction",
principal: new iam.ServicePrincipal("s3.amazonaws.com"),
sourceArn: props.bucket.bucketArn,
});
// don't create the notification custom-resource until after both the bucket and lambda
// are fully created and policies applied.
notificationResource.node.addDependency(props.bucket);
notificationResource.node.addDependency(props.lambda);
}
}
答案 5 :(得分：0)
感谢上面的精彩回答，有关s3-> lambda通知的构造，请参见下文。可以像
一样使用 const fn = new SingletonFunction(this, "Function", {
...
});
const bucket = Bucket.fromBucketName(this, "Bucket", "...");
const s3notification = new S3NotificationLambda(this, "S3Notification", {
bucket: bucket,
lambda: function,
events: ['s3:ObjectCreated:*'],
prefix: "some_prefix/"
})
构造（以.ts文件的形式插入您的项目）
import * as cr from "@aws-cdk/custom-resources";
import * as logs from "@aws-cdk/aws-logs";
import * as s3 from "@aws-cdk/aws-s3";
import * as sqs from "@aws-cdk/aws-sqs";
import * as iam from "@aws-cdk/aws-iam";
import { Construct } from "@aws-cdk/core";
import * as lambda from "@aws-cdk/aws-lambda";
export interface S3NotificationLambdaProps {
bucket: s3.IBucket;
lambda: lambda.IFunction;
events: string[];
prefix: string;
}
export class S3NotificationLambda extends Construct {
constructor(scope: Construct, id: string, props: S3NotificationLambdaProps) {
super(scope, id);
const notificationResource = new cr.AwsCustomResource(
scope,
id + "CustomResource",
{
onCreate: {
service: "S3",
action: "putBucketNotificationConfiguration",
parameters: {
// This bucket must be in the same region you are deploying to
Bucket: props.bucket.bucketName,
NotificationConfiguration: {
LambdaFunctionConfigurations: [
{
Events: props.events,
LambdaFunctionArn: props.lambda.functionArn,
Filter: {
Key: {
FilterRules: [{ Name: "prefix", Value: props.prefix }],
},
},
},
],
},
},
physicalResourceId: <cr.PhysicalResourceId>(
(id + Date.now().toString())
),
},
onDelete: {
service: "S3",
action: "putBucketNotificationConfiguration",
parameters: {
// This bucket must be in the same region you are deploying to
Bucket: props.bucket.bucketName,
// deleting a notification configuration involves setting it to empty.
NotificationConfiguration: {},
},
physicalResourceId: <cr.PhysicalResourceId>(
(id + Date.now().toString())
),
},
policy: cr.AwsCustomResourcePolicy.fromStatements([
new iam.PolicyStatement({
// The actual function is PutBucketNotificationConfiguration.
// The "Action" for IAM policies is PutBucketNotification.
// https://docs.aws.amazon.com/AmazonS3/latest/dev/list_amazons3.html#amazons3-actions-as-permissions
actions: ["S3:PutBucketNotification", "S3:GetBucketNotification"],
// allow this custom resource to modify this bucket
resources: [props.bucket.bucketArn],
}),
]),
}
);
props.lambda.addPermission("AllowS3Invocation", {
action: "lambda:InvokeFunction",
principal: new iam.ServicePrincipal("s3.amazonaws.com"),
sourceArn: props.bucket.bucketArn,
});
// don't create the notification custom-resource until after both the bucket and queue
// are fully created and policies applied.
notificationResource.node.addDependency(props.bucket);
notificationResource.node.addDependency(props.lambda);
}
}
答案 6 :(得分：0)
我使用 CloudTrail 解决了这个问题，代码如下所示，而且更抽象：
const trail = new cloudtrail.Trail(this, 'MyAmazingCloudTrail');
const options: AddEventSelectorOptions = {
readWriteType: cloudtrail.ReadWriteType.WRITE_ONLY
};
// Adds an event selector to the bucket
trail.addS3EventSelector([{
bucket: bucket, // 'Bucket' is of type s3.IBucket,
}], options);
bucket.onCloudTrailWriteObject('MyAmazingCloudTrail', {
target: new targets.LambdaFunction(functionReference)
});