Question

我正在尝试运行一个cloudformation堆栈来创建一个EMR集群，并将一个负载均衡器连接到该EMR的主节点。

要生成EMR集群，我在堆栈中使用以下代码：

"EmrCluster": {
      "Type": "AWS::EMR::Cluster",
      "DependsOn": "nat0a30a49f0c2913dc4",
      "Properties": {
        "Name": "EmrCluster",
        "Applications": [
          {
            "Name": "Spark"
          },
          {
            "Name": "Hive"
          },
          {
            "Name": "Hadoop"
          },
          {
            "Name": "Livy"
          },
          {
            "Name": "Hue"
          },
          {
            "Name": "Zeppelin"
          }
        ],
        "Instances": {
          "AdditionalMasterSecurityGroups": [
              {
                "Ref": "sgMaster"
              }
          ],
          "Ec2KeyName": {"Ref": "KeyName"},
          "Ec2SubnetId": {"Ref": "subnet03ead9bfe3352d90b"},
          "MasterInstanceGroup": {
            "InstanceCount": 1,
            "InstanceType": "m3.xlarge",
            "Name": "Master",
          },
          "CoreInstanceGroup": { 
            "InstanceCount": 2,
            "InstanceType": "m3.xlarge",
            "Name": "Core",
          }
        },
        "Configurations": [
          {
            "Classification": "spark-env",
            "Configurations": [
              {
                "Classification": "export",
                "ConfigurationProperties": {
                  "PYSPARK_PYTHON": "/usr/bin/python3"
                }
              }
            ]
          }
        ],
        "JobFlowRole": "EMR_EC2_DefaultRole",
        "ServiceRole": "EMR_DefaultRole",
        "ReleaseLabel": "emr-5.13.0"
      }
    },

为了将负载均衡器连接到上述集群的主节点，我需要在以下代码中添加主节点的instanceID：

"loadBalancer" : {
      "Type": "AWS::ElasticLoadBalancingV2::LoadBalancer",
      "Properties": {
        "Subnets" : [ {"Ref": "subnet04a1fe03c3f3c42fd"}, {"Ref" : "subnet03ead9bfe3352d90b"} ]
      }
    },
    "TargetGroup" : {
      "Type" : "AWS::ElasticLoadBalancingV2::TargetGroup",
      "Properties" : {
        "HealthCheckIntervalSeconds": 30,
        "HealthCheckProtocol": "HTTP",
        "HealthCheckTimeoutSeconds": 10,
        "HealthyThresholdCount": 5,
        "Matcher" : {
          "HttpCode" : "200"
        },
        "Name": "MyTargets",
        "Port": 8890,
        "Protocol": "HTTP",
        "Targets": [
          { "Id": {"Ref" : "MasternodeID"}, "Port": 8890 }
        ],
        "UnhealthyThresholdCount": 3,
        "VpcId": {"Ref" : "vpc07164705742b384fb"}
      }
    }

如何找到并传递EMR集群主节点的instanceID作为＆＃34; MasternodeID＆＃34;在目标群体中。简单来说，如何获取在同一堆栈中创建的EMR集群的主节点的instanceID。

Answer 1

由于可用云形式提供公共主DNS，因此我已经从对我有用的DNS中获取了ip

EMRMasterTargetGroup:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
DependsOn: EMRELB
Properties:
  Name: emr-master-target-group
  Protocol: HTTP
  Port: 8088
  VpcId: !Ref VPCId
  HealthCheckPort: 8088
  HealthCheckProtocol: HTTP
  HealthCheckPath: /cluster
  TargetType: ip
  Targets:
    - Id:  !Join
            - '.'
            - - !Select [1, !Split ['-', !GetAtt EMRCluster.MasterPublicDNS]]
              - !Select [2, !Split ['-', !GetAtt EMRCluster.MasterPublicDNS]]
              - !Select [3, !Split ['-', !GetAtt EMRCluster.MasterPublicDNS]]
              - !Select [0, !Split ['.', !Select [4, !Split ['-', !GetAtt EMRCluster.MasterPublicDNS]]]]
      Port: 8088
  Tags:
    - Key: Name
      Value: emr-master-target-group

如何在cloudformation脚本中引用EMR集群的主实例？

1 个答案: