我有一个基于https://aws.amazon.com/blogs/compute/bluegreen-deployments-with-amazon-ecs/在Terraform中构建的蓝绿色ECS群集。那篇文章和相关的Github repo没有提到自动缩放。
我正在努力让Application Autoscaling正常工作,但我没有看到我的EC2实例数量上升或下降。目前我正在使用目标跟踪ECSServiceAverageCPUUtilization,目标值为15.换句话说,我希望ECS在群集中的每个EC2实例上维持15%的平均CPU利用率。如果平均CPU利用率攀升,则自动缩放应该向外扩展以使平均值回落;如果CPU平均值下降,则群集应该缩小。(15%太低,但我试图更容易地进行缩放。)
实际CPU利用率高于50%,因此不会进行扩展。我怀疑主动(蓝色)和非主动(绿色)服务相互决斗。
如何在ECS上使用Blue-Green群集获得Autoscaling,Application或Classic?我喜欢目标跟踪的想法,但我对其他想法持开放态度。
扩展政策:
$ aws application-autoscaling describe-scaling-policies --service-namespace ecs \
--policy-names hub-prod_blue_cpu_target_tracking hub-prod_green_cpu_target_tracking
{
"ScalingPolicies": [
{
"PolicyName": "hub-prod_blue_cpu_target_tracking",
"ScalableDimension": "ecs:service:DesiredCount",
"ResourceId": "service/hub-prod_ecs_cluster/hub-prod_es_blue",
"CreationTime": 1519243876.638,
"PolicyARN": "arn:aws:autoscaling:us-east-1:ORGID:scalingPolicy:db09a76b-f260-4699-b5aa-cb837086b87d:resource/ecs/service/hub-prod_ecs_cluster/hub-prod_es_blue:policyName/hub-prod_blue_cpu_target_tracking",
"PolicyType": "TargetTrackingScaling",
"TargetTrackingScalingPolicyConfiguration": {
"TargetValue": 15.0,
"PredefinedMetricSpecification": {
"PredefinedMetricType": "ECSServiceAverageCPUUtilization"
},
"ScaleOutCooldown": 300,
"DisableScaleIn": false,
"ScaleInCooldown": 300
},
"Alarms": [
{
"AlarmName": "TargetTracking-service/hub-prod_ecs_cluster/hub-prod_es_blue-AlarmHigh-768f842f-5811-4856-bccf-d73b91f20bfb",
"AlarmARN": "arn:aws:cloudwatch:us-east-1:ORGID:alarm:TargetTracking-service/hub-prod_ecs_cluster/hub-prod_es_blue-AlarmHigh-768f842f-5811-4856-bccf-d73b91f20bfb"
},
{
"AlarmName": "TargetTracking-service/hub-prod_ecs_cluster/hub-prod_es_blue-AlarmLow-d710adc7-5414-46a5-8222-fde9d816f4ee",
"AlarmARN": "arn:aws:cloudwatch:us-east-1:ORGID:alarm:TargetTracking-service/hub-prod_ecs_cluster/hub-prod_es_blue-AlarmLow-d710adc7-5414-46a5-8222-fde9d816f4ee"
}
],
"ServiceNamespace": "ecs"
},
{
"PolicyName": "hub-prod_green_cpu_target_tracking",
"ScalableDimension": "ecs:service:DesiredCount",
"ResourceId": "service/hub-prod_ecs_cluster/hub-prod_es_green",
"CreationTime": 1519243876.577,
"PolicyARN": "arn:aws:autoscaling:us-east-1:ORGID:scalingPolicy:a407d322-c2a1-4363-9be9-1833f6d3dfce:resource/ecs/service/hub-prod_ecs_cluster/hub-prod_es_green:policyName/hub-prod_green_cpu_target_tracking",
"PolicyType": "TargetTrackingScaling",
"TargetTrackingScalingPolicyConfiguration": {
"TargetValue": 15.0,
"PredefinedMetricSpecification": {
"PredefinedMetricType": "ECSServiceAverageCPUUtilization"
},
"ScaleOutCooldown": 300,
"DisableScaleIn": false,
"ScaleInCooldown": 300
},
"Alarms": [
{
"AlarmName": "TargetTracking-service/hub-prod_ecs_cluster/hub-prod_es_green-AlarmHigh-d7a722a6-b7ab-4bad-846f-d728d4c03763",
"AlarmARN": "arn:aws:cloudwatch:us-east-1:ORGID:alarm:TargetTracking-service/hub-prod_ecs_cluster/hub-prod_es_green-AlarmHigh-d7a722a6-b7ab-4bad-846f-d728d4c03763"
},
{
"AlarmName": "TargetTracking-service/hub-prod_ecs_cluster/hub-prod_es_green-AlarmLow-074e9dde-7936-4b53-ba31-0665190c084f",
"AlarmARN": "arn:aws:cloudwatch:us-east-1:ORGID:alarm:TargetTracking-service/hub-prod_ecs_cluster/hub-prod_es_green-AlarmLow-074e9dde-7936-4b53-ba31-0665190c084f"
}
],
"ServiceNamespace": "ecs"
}
]
}
可扩展目标:
$ aws application-autoscaling describe-scalable-targets --service-namespace ecs \
--resource-ids service/hub-prod_ecs_cluster/hub-prod_es_blue service/hub-prod_ecs_cluster/hub-prod_es_green
{
"ScalableTargets": [
{
"ScalableDimension": "ecs:service:DesiredCount",
"ResourceId": "service/hub-prod_ecs_cluster/hub-prod_es_blue",
"RoleARN": "arn:aws:iam::ORGID:role/aws-service-role/ecs.application-autoscaling.amazonaws.com/AWSServiceRoleForApplicationAutoScaling_ECSService",
"CreationTime": 1519240496.0,
"MinCapacity": 3,
"ServiceNamespace": "ecs",
"MaxCapacity": 30
},
{
"ScalableDimension": "ecs:service:DesiredCount",
"ResourceId": "service/hub-prod_ecs_cluster/hub-prod_es_green",
"RoleARN": "arn:aws:iam::ORGID:role/aws-service-role/ecs.application-autoscaling.amazonaws.com/AWSServiceRoleForApplicationAutoScaling_ECSService",
"CreationTime": 1519167581.233,
"MinCapacity": 3,
"ServiceNamespace": "ecs",
"MaxCapacity": 30
}
]
}
扩展活动。 (1519252380 = 2018-02-22T2230Z,就在我将ALB Listener的默认目标组修改为蓝色之前。)
蓝:
$ aws application-autoscaling describe-scaling-activities --service-namespace ecs \
--resource-id service/hub-prod_ecs_cluster/hub-prod_es_blue \
| jq '.ScalingActivities | .[] | select(.StartTime >= 1519252380)'
{
"ScalableDimension": "ecs:service:DesiredCount",
"Description": "Setting desired count to 24.",
"ResourceId": "service/hub-prod_ecs_cluster/hub-prod_es_blue",
"ActivityId": "db73de6d-3918-4f24-94b7-de7437c767e4",
"StartTime": 1519252897.326,
"ServiceNamespace": "ecs",
"Cause": "monitor alarm TargetTracking-service/hub-prod_ecs_cluster/hub-prod_es_blue-AlarmHigh-768f842f-5811-4856-bccf-d73b91f20bfb in state ALARM triggered policy hub-prod_blue_cpu_target_tracking",
"StatusMessage": "Successfully set desired count to 24. Waiting for change to be fulfilled by ecs.",
"StatusCode": "InProgress"
}
{
"ScalableDimension": "ecs:service:DesiredCount",
"Description": "Setting desired count to 21.",
"ResourceId": "service/hub-prod_ecs_cluster/hub-prod_es_blue",
"ActivityId": "dce5efc9-2c93-4799-87bb-0d344a917de8",
"StartTime": 1519252837.315,
"ServiceNamespace": "ecs",
"EndTime": 1519252907.177,
"Cause": "monitor alarm TargetTracking-service/hub-prod_ecs_cluster/hub-prod_es_blue-AlarmHigh-768f842f-5811-4856-bccf-d73b91f20bfb in state ALARM triggered policy hub-prod_blue_cpu_target_tracking",
"StatusMessage": "Successfully set desired count to 21. Found it was later changed to 24.",
"StatusCode": "Overridden"
}
{
"ScalableDimension": "ecs:service:DesiredCount",
"Description": "Setting desired count to 18.",
"ResourceId": "service/hub-prod_ecs_cluster/hub-prod_es_blue",
"ActivityId": "e235d7c8-3558-4648-a276-5cb81f1b7af2",
"StartTime": 1519252777.465,
"ServiceNamespace": "ecs",
"EndTime": 1519252843.723,
"Cause": "monitor alarm TargetTracking-service/hub-prod_ecs_cluster/hub-prod_es_blue-AlarmHigh-768f842f-5811-4856-bccf-d73b91f20bfb in state ALARM triggered policy hub-prod_blue_cpu_target_tracking",
"StatusMessage": "Successfully set desired count to 18. Found it was later changed to 21.",
"StatusCode": "Overridden"
}
{
"ScalableDimension": "ecs:service:DesiredCount",
"Description": "Setting desired count to 13.",
"ResourceId": "service/hub-prod_ecs_cluster/hub-prod_es_blue",
"ActivityId": "47835aee-9c64-428d-81b3-0367da773370",
"StartTime": 1519252717.297,
"ServiceNamespace": "ecs",
"EndTime": 1519252784.681,
"Cause": "monitor alarm TargetTracking-service/hub-prod_ecs_cluster/hub-prod_es_blue-AlarmHigh-768f842f-5811-4856-bccf-d73b91f20bfb in state ALARM triggered policy hub-prod_blue_cpu_target_tracking",
"StatusMessage": "Successfully set desired count to 13. Found it was later changed to 18.",
"StatusCode": "Overridden"
}
绿色:
$ aws application-autoscaling describe-scaling-activities --service-namespace ecs \
--resource-id service/hub-prod_ecs_cluster/hub-prod_es_green \
| jq '.ScalingActivities | .[] | select(.StartTime >= 1519252380)'
{
"ScalableDimension": "ecs:service:DesiredCount",
"Description": "Setting desired count to 3.",
"ResourceId": "service/hub-prod_ecs_cluster/hub-prod_es_green",
"ActivityId": "4b910f88-87da-4173-899e-246533c4d35c",
"StartTime": 1519253491.575,
"ServiceNamespace": "ecs",
"EndTime": 1519253558.285,
"Cause": "monitor alarm TargetTracking-service/hub-prod_ecs_cluster/hub-prod_es_green-AlarmLow-074e9dde-7936-4b53-ba31-0665190c084f in state ALARM triggered policy hub-prod_green_cpu_target_tracking",
"StatusMessage": "Successfully set desired count to 3. Change successfully fulfilled by ecs.",
"StatusCode": "Successful"
}