Question

尽管我的研究告诉我这是一个内部到亚马逊的错误，但我仍然一直得到这个错误。我不知道从哪里开始这个错误，或者甚至有什么我可以做的来帮助它。

事实上，我一直这样做，这让我觉得我的剧本出了问题。这是：

{
  "Description": "Demo pipeline.",
  "Resources": {
    "s3Demo": {
        "Type" : "AWS::S3::Bucket",
        "Properties" : {
            "BucketName" : "example-dna-demo"
        }
    },

    "s3Access": {
        "Type": "AWS::IAM::Role",
        "Properties": {
            "ManagedPolicyArns": [
                "arn:aws:iam::aws:policy/AmazonS3FullAccess"
            ],
            "AssumeRolePolicyDocument": {
                "Version": "2012-10-17",
                "Statement": [{
                    "Effect": "Allow",
                    "Action": "sts:AssumeRole",
                    "Principal":{
                        "Service": "firehose.amazonaws.com"
                    }
                }]
            }, 
            "RoleName": "kinesisS3Access"
        },
        "DependsOn": "s3Demo"
    },

    "kinesisDemo": {
        "Type": "AWS::KinesisFirehose::DeliveryStream",
        "Properties": {
            "DeliveryStreamName": "Demo-Stream",
            "S3DestinationConfiguration": {
                "BucketARN" : "arn:aws:s3:::example-dna-demo",
                "BufferingHints" : {
                    "IntervalInSeconds" : 300,
                    "SizeInMBs" : 5
                },
                "CompressionFormat" : "UNCOMPRESSED",
                "Prefix" : "twitter",
                "RoleARN" : { "Fn::GetAtt": [ "s3Access", "Arn" ]}
            }
        },
        "DependsOn": "s3Access"
    },

    "S3LambdaAccess":{
        "Type": "AWS::IAM::Role",
        "Properties": {
            "ManagedPolicyArns": [
                "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess"
            ],
            "AssumeRolePolicyDocument": {
                "Version": "2012-10-17",
                "Statement": [{
                    "Effect": "Allow",
                    "Action": "sts:AssumeRole",
                    "Principal":{
                        "Service": "lambda.amazonaws.com"
                    }
                }]
            }, 
            "RoleName": "lambdaS3Access"
        }
    },
    "LambdaDemo": {
        "Type" : "AWS::Lambda::Function",
        "Properties" : {
            "Code" : {
                "S3Bucket" : "example-dna-cloud-formation",
                "S3Key" : "lambda_function.py.zip"
            },
            "Description" : "Looks for S3 writes and loads them into another resource",
            "FunctionName" : "DemoLambdaFunction",
            "Handler" : "lambda-handler",
            "Role" : { "Fn::GetAtt": [ "S3LambdaAccess", "Arn" ]},
            "Runtime" : "python2.7"
        },
        "DependsOn": "S3LambdaAccess"
    },
    "EMRClusterJobFlowRole": {
        "Type": "AWS::IAM::Role",
        "Properties": {
            "AssumeRolePolicyDocument": {  
                "Version": "2012-10-17",
                "Statement": [{
                    "Effect": "Allow",
                    "Action": "sts:AssumeRole",
                    "Principal":{
                        "Service": "ec2.amazonaws.com"
                    }
                }] 
            },
            "RoleName": "ClusterRole"
        }
    },
    "EMRServiceRole": {
        "Type": "AWS::IAM::Role",
        "Properties": {
            "AssumeRolePolicyDocument": { 
              "Version": "2012-10-17",
                "Statement": [{
                    "Effect": "Allow",
                    "Action": "sts:AssumeRole",
                    "Principal":{
                        "Service": "ec2.amazonaws.com"
                    }
                }]
            },
            "RoleName": "EC2InstanceRole"
        }
    },
    "EMR":{
        "Type" : "AWS::EMR::Cluster",
        "Properties" : {
            "Applications": [
                {
                    "Name" : "Spark"
                }
            ],
            "ReleaseLabel": "emr-5.0.0",
            "Instances" : {
                "CoreInstanceGroup" : {
                    "BidPrice": 0.06,
                    "InstanceCount" : 1,
                    "InstanceType" : "m4.large",
                    "Market": "SPOT"
                    },
                "MasterInstanceGroup" : {
                    "BidPrice": 0.06,
                    "InstanceCount" : 1,
                    "InstanceType" : "m4.large",
                    "Market": "SPOT"
                    }
                },
            "JobFlowRole" : "EMRClusterJobFlowRole",
            "Name" : "DemoEMR",
            "ServiceRole" : "EMRServiceRole",
            "LogUri":"s3://toyota-dna-cloud-formation/cf-logging"
        },
        "DependsOn": ["EMRServiceRole", "EMRServiceRole"]
    }
  }
}

我想你可能无法运行它，因为我有一个lambda函数从S3存储桶获取代码，我已经改变了这里的名字。我只是在学习云形成脚本，我知道有很多东西我不在这里做，但我只想构建一个有用的小东西，然后再填充一点。

我知道我的脚本一直工作到两个IAM角色和EMR集群。提前谢谢。

编辑：我指定了最近的实例版本并选择了ReleaseLabel属性。没有运气。同样的错误。

Answer 1

可能是您的帐户已达到您尝试部署到的区域中的EC2限制。你尝试过不同的地区吗？

Answer 2

事实证明，我在运行脚本的区域中没有默认的VPC，这就是我的EMR集群无法稳定的原因。

当我尝试在不同的区域运行它时，它可以工作，但因为该区域DID有一个默认的VPC。

Answer 3

在我的情况下，由于缺少自动缩放角色，称为EMR_AutoScaling_DefaultRole。

一旦我通过aws emr create-default-roles将其安装到位，我的cloudformation堆栈再次开始部署得很好（它在我添加自动缩放内容之前部署好了）。

AWS Cloud Formation错误：ElasticMapReduce Cluster无法稳定

3 个答案: