我正在尝试构建一个Kibana仪表板,其中包含通过AWS Kinesis firehose收集的Twitter数据,其中数据传递到S3存储桶,该存储桶触发Lambda函数,该函数将数据传递给AWS Elastic Search,然后传递给Kibana。我正在关注此博客https://aws.amazon.com/blogs/big-data/building-a-near-real-time-discovery-platform-with-aws/
数据正确加载到S3存储桶中,但它永远不会到达Kibana,我相信这是因为Lambda函数没有被S3中的事件触发,因为我希望(没有调用或日志)。我想这是因为我没有正确设置权限。 Lambda函数可以由测试事件手动调用。
在Lambda函数页面上,我选择了一个名为lambda_s3_exec_role的现有角色,该角色附有AWSLambdaExecute策略,但我觉得我错过了S3更具体的其他内容。我无法在创建lambda函数部分的博客中关注此行,因为我无法识别这些选项:
" 10。选择lambda_s3_exec_role(如果此值不存在,请选择Create new role S3 execution role)。"
任何人都可以帮我为Lambda函数创建适当的角色/策略,或者发现可能出现的问题吗?
从我目前拥有的Lambda函数的查看权限:
功能政策
{
"Version": "2012-10-17",
"Id": "default",
"Statement": [
{
"Sid": "****",
"Effect": "Allow",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Action": "lambda:InvokeFunction",
"Resource": "****",
"Condition": {
"ArnLike": {
"AWS:SourceArn": "arn:aws:s3:::****"
}
}
}
]
}
执行角色
{
"roleName": "lambda_s3_exec_role",
"policies": [
{
"document": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:*"
],
"Resource": "arn:aws:logs:*:*:*"
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::*"
}
]
},
"name": "AWSLambdaExecute",
"id": "****",
"type": "managed",
"arn": "arn:aws:iam::aws:policy/AWSLambdaExecute"
}
]
}
答案 0 :(得分:3)
您列出的权限看起来不错,因此我将尝试提供一些可能有助于查找问题的步骤,因为很难明确了解您的问题所在。
lambda.amazonaws.com
twitter/raw-data/
。如果firehose正在写入不是事件前缀的路径,则不会调用该事件。aws lambda invoke \
--invocation-type RequestResponse \
--function-name helloworld \
--region region \
--log-type Tail \
--payload file://dummy_event.json \
--profile adminuser \
outputfile.txt
<强> dummy_event.json 强>
{
"Records":[
{
"eventVersion":"2.0",
"eventSource":"aws:s3",
"awsRegion":"us-west-2",
"eventTime":"1970-01-01T00:00:00.000Z",
"eventName":"ObjectCreated:Put",
"userIdentity":{
"principalId":"AIDAJDPLRKLG7UEXAMPLE"
},
"requestParameters":{
"sourceIPAddress":"127.0.0.1"
},
"responseElements":{
"x-amz-request-id":"C3D13FE58DE4C810",
"x-amz-id-2":"FMyUVURIY8/IgAtTv8xRjskZQpcIZ9KG4V5Wp6S7S/JRWeUWerMUE5JgHvANOjpD"
},
"s3":{
"s3SchemaVersion":"1.0",
"configurationId":"testConfigRule",
"bucket":{
"name":"sourcebucket",
"ownerIdentity":{
"principalId":"A3NL1KOZZKExample"
},
"arn":"arn:aws:s3:::sourcebucket"
},
"object":{
"key":"HappyFace.jpg",
"size":1024,
"eTag":"d41d8cd98f00b204e9800998ecf8427e",
"versionId":"096fKKXTRTtl3on89fVO.nfljtsv6qko"
}
}
}
]
}