运行AWS ECS容器的ELB运行状况检查失败

时间:2020-01-16 01:07:44

标签: amazon-web-services containers amazon-cloudformation amazon-ecs amazon-elb

我目前正在尝试通过CloudFormation模板将应用程序部署到AWS ECS上。 docker映像存储在AWS ECR中,并部署到由Application Load Balancer开头的ECS服务中。

我的服务启动,并创建了负载均衡器,但是ECS服务内部的任务反复失败,并显示以下错误:

Task failed ELB health checks in (target-group arn:aws:elasticloadbalancing:us-east-1:...

我已经检查了我的安全组-ECS服务安全组包括负载均衡器安全组,并且负载均衡器已成功创建。

我手动尝试了在ECR上提取我的映像并运行它-那里没有问题。我想念什么?我的模板在下面。


Resources:
  ECSRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Effect: Allow
          Principal:
            Service: [ecs.amazonaws.com]
          Action: ['sts:AssumeRole']
      Path: /
      Policies:
      - PolicyName: ecs-service
        PolicyDocument:
          Statement:
          - Effect: Allow
            Action:
              # Rules which allow ECS to attach network interfaces to instances
              # on your behalf in order for awsvpc networking mode to work right
              - 'ec2:AttachNetworkInterface'
              - 'ec2:CreateNetworkInterface'
              - 'ec2:CreateNetworkInterfacePermission'
              - 'ec2:DeleteNetworkInterface'
              - 'ec2:DeleteNetworkInterfacePermission'
              - 'ec2:Describe*'
              - 'ec2:DetachNetworkInterface'

              # Rules which allow ECS to update load balancers on your behalf
              # with the information sabout how to send traffic to your containers
              - 'elasticloadbalancing:DeregisterInstancesFromLoadBalancer'
              - 'elasticloadbalancing:DeregisterTargets'
              - 'elasticloadbalancing:Describe*'
              - 'elasticloadbalancing:RegisterInstancesWithLoadBalancer'
              - 'elasticloadbalancing:RegisterTargets'
            Resource: '*'

  # This is a role which is used by the ECS tasks themselves.
  ECSTaskExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Effect: Allow
          Principal:
            Service: [ecs-tasks.amazonaws.com]
          Action: ['sts:AssumeRole']
      Path: /
      Policies:
        - PolicyName: AmazonECSTaskExecutionRolePolicy
          PolicyDocument:
            Statement:
            - Effect: Allow
              Action:
                # Allow the ECS Tasks to download images from ECR
                - 'ecr:GetAuthorizationToken'
                - 'ecr:BatchCheckLayerAvailability'
                - 'ecr:GetDownloadUrlForLayer'
                - 'ecr:BatchGetImage'

                # Allow the ECS tasks to upload logs to CloudWatch
                - 'logs:CreateLogStream'
                - 'logs:PutLogEvents'
              Resource: '*'
  TaskDef:
    Type: AWS::ECS::TaskDefinition
    Properties: 
      Cpu: 4096    
      Memory: 30720
      ContainerDefinitions: 
        - Image: !Ref ECRImageUrl
          Name: !Sub "${ProjectName}-ecsContainer"
          PortMappings: 
            - ContainerPort: 4000
              HostPort: 4000
              Protocol: tcp
      Family: !Sub "${ProjectName}-taskDef"
      ExecutionRoleArn: !Ref ECSTaskExecutionRole
      RequiresCompatibilities: 
        - FARGATE
      NetworkMode: awsvpc

  Cluster:
    Type: AWS::ECS::Cluster
    Properties:
      ClusterName: !Sub "${ProjectName}-ECSCluster"

  Service:
    Type: AWS::ECS::Service
    DependsOn:
      - LoadBalancerListener
    Properties:
      Cluster: !Ref Cluster
      DesiredCount: 2
      LaunchType: FARGATE
      ServiceName: !Sub "${ProjectName}-ECSService"
      TaskDefinition: !Ref TaskDef
      NetworkConfiguration:
        AwsvpcConfiguration:
          SecurityGroups: 
            - !Ref FargateContainerSecurityGroup
          AssignPublicIp: ENABLED
          Subnets: !Split [',', {'Fn::ImportValue': !Sub '${VPCStackName}-PublicSubnets'}]
      LoadBalancers:
        - ContainerName: !Sub "${ProjectName}-ecsContainer"
          ContainerPort: 4000
          TargetGroupArn: !Ref TargetGroup

  FargateContainerSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Access to the Fargate containers
      VpcId:
        Fn::ImportValue: 
          !Sub '${VPCStackName}-VPC'
  EcsSecurityGroupIngressFromPublicALB:
    Type: AWS::EC2::SecurityGroupIngress
    Properties:
      Description: Ingress from the public ALB
      GroupId: !Ref 'FargateContainerSecurityGroup'
      IpProtocol: -1
      SourceSecurityGroupId: !Ref 'PublicLoadBalancerSG'
  EcsSecurityGroupIngressFromSelf:
    Type: AWS::EC2::SecurityGroupIngress
    Properties:
      Description: Ingress from other containers in the same security group
      GroupId: !Ref 'FargateContainerSecurityGroup'
      IpProtocol: -1
      SourceSecurityGroupId: !Ref 'FargateContainerSecurityGroup'
  PublicLoadBalancerSG:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Access to the public facing load balancer
      VpcId: 
        Fn::ImportValue: 
          !Sub '${VPCStackName}-VPC'
      SecurityGroupIngress:
          - CidrIp: 0.0.0.0/0
            IpProtocol: -1

  ACMCertificate:
    Type: AWS::CertificateManager::Certificate
    Properties: 
      DomainName: !Sub ${ProjectName}.${DomainName}
      ValidationMethod: DNS

  TargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    DependsOn:
      - LoadBalancer
    Properties:
      TargetType: ip
      Name: !Sub "${ProjectName}-ECSService"
      Port: 4000
      Protocol: HTTP
      VpcId: 
        Fn::ImportValue: 
          !Sub '${VPCStackName}-VPC'

  LoadBalancer:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Scheme: internet-facing
      Subnets: !Split [',', {'Fn::ImportValue': !Sub '${VPCStackName}-PublicSubnets'}]
      SecurityGroups: 
        - !Ref PublicLoadBalancerSG

  LoadBalancerListener:
    Type: AWS::ElasticLoadBalancingV2::Listener
    DependsOn:
      - LoadBalancer
    Properties:
      DefaultActions:
        - TargetGroupArn: !Ref TargetGroup
          Type: 'forward'
      LoadBalancerArn: !Ref LoadBalancer
      Port: 443
      Protocol: HTTP

3 个答案:

答案 0 :(得分:3)

事实证明,我的安全组不够宽松。来自网络负载平衡器的流量被视为来自其原始来源,因此,如果您的NLB对所有流量开放,那么Fargate容器也应如此。这解决了我的问题:


  FargateContainerSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Access to the Fargate containers
      VpcId:
        Fn::ImportValue: 
          !Sub '${VPCStackName}-VPC'
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: !Ref ApplicationPort
          ToPort: !Ref ApplicationPort
          CidrIp: 0.0.0.0/0

答案 1 :(得分:1)

运行状况检查功能会自动在端口80上呼叫/并期望200状态码作为响应。它在EC2->目标组->您的ecs目标组中可用。您必须确保端口为4000,并在运行状况检查中调整默认路径和响应状态代码。

此外,您始终可以尝试使用正在使用的端口4000上的公共ip或DNS连接到ec2实例,并查看其是否有效。

如果ec2实例在端口4000上不工作,则对docker部署进行故障排除。会谈定义或参数有问题。

如果目标组扩展或运行状况检查配置有问题,则为

希望这会有所帮助。

答案 2 :(得分:0)

在经历了很多痛苦和磨难之后,我发现 ALB 本身需要与安全组 (SG) 相关联,以允许 ECS 动态分配的端口上的流量。您应该自动定义一个 SG 来定义这些端口范围。将此 SG 与您的 ALB 相关联,您的健康检查将开始通过(假设其他一切都已正确连接)。

此外,确保您的任务定义将网络模式设置为“桥接”并且将“hostPort”值设置为 0——这表明 ECS 在底层 EC2 实例上动态分配端口并将其映射到您的集装箱港口。