创建AMI图像作为云形成堆栈的一部分

时间:2014-01-29 12:40:53

标签: amazon-ec2 ami amazon-cloudformation

我想创建一个EC2 cloudformation堆栈,基​​本上可以按照以下步骤进行描述:

1.-启动实例

2.-提供实例

3.-停止实例并从中创建AMI图像

4.-创建一个自动缩放组,将创建的AMI图像作为源来启动新实例。

基本上我可以在一个cloudformation模板中执行1和2,在第二个模板中执行4。我似乎无法做的是从云信息模板中的实例创建AMI图像,如果我想删除堆栈,基本上会产生必须手动删除AMI的问题。

话虽如此,我的问题是:

1.-有没有办法从云形态模板中的实例创建AMI图像?

2.-如果1的答案为否,是否有办法添加AMI图像(或任何其他资源)以使其成为完整堆栈的一部分?

编辑:

为了澄清一下,我已经解决了创建AMI并在云信息模板中使用它的问题,我无法在云形态模板中创建AMI INSIDE或以某种方式将其添加到创建的堆栈中。

正如我对Rico的答案所评论的那样,我现在所做的是使用一个基本上有3个步骤的ansible playbook:

1.-使用云形态模板创建基本实例

2.-使用ansible创建在步骤1中创建的实例的AMI

3.-使用第二个云形式模板创建堆栈的其余部分(ELB,自动调节组等),该模板更新在步骤1中创建的模板,并使用在步骤2中创建的AMI来启动实例。

这就是我现在管理它的方式,但我想知道是否有任何方法可以创建AMI INSIDE一个云形态模板,或者是否可以将创建的AMI添加到堆栈中(类似于告诉堆栈,“嘿,这也属于你,所以处理它“)。

4 个答案:

答案 0 :(得分:33)

是的,您可以在CloudFormation模板中的EC2实例中创建AMI,方法是实现在创建时调用Custom Resource API的CreateImage(并调用DeregisterImage和{{3删除时的API)。

由于AMI有时需要很长时间才能创建,因此如果在Lambda函数超时之前等待尚未完成,则Lambda支持的自定义资源将需要重新调用自身。

这是一个完整的例子:

DeleteSnapshot

Description: Create an AMI from an EC2 instance.
Parameters:
  ImageId:
    Description: Image ID for base EC2 instance.
    Type: AWS::EC2::Image::Id
    # amzn-ami-hvm-2016.09.1.20161221-x86_64-gp2
    Default: ami-9be6f38c
  InstanceType:
    Description: Instance type to launch EC2 instances.
    Type: String
    Default: m3.medium
    AllowedValues: [ m3.medium, m3.large, m3.xlarge, m3.2xlarge ]
Resources:
  # Completes when the instance is fully provisioned and ready for AMI creation.
  AMICreate:
    Type: AWS::CloudFormation::WaitCondition
    CreationPolicy:
      ResourceSignal:
        Timeout: PT10M
  Instance:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: !Ref ImageId
      InstanceType: !Ref InstanceType
      UserData:
        "Fn::Base64": !Sub |
          #!/bin/bash -x
          yum -y install mysql # provisioning example
          /opt/aws/bin/cfn-signal \
            -e $? \
            --stack ${AWS::StackName} \
            --region ${AWS::Region} \
            --resource AMICreate
          shutdown -h now
  AMI:
    Type: Custom::AMI
    DependsOn: AMICreate
    Properties:
      ServiceToken: !GetAtt AMIFunction.Arn
      InstanceId: !Ref Instance
  AMIFunction:
    Type: AWS::Lambda::Function
    Properties:
      Handler: index.handler
      Role: !GetAtt LambdaExecutionRole.Arn
      Code:
        ZipFile: !Sub |
          var response = require('cfn-response');
          var AWS = require('aws-sdk');
          exports.handler = function(event, context) {
            console.log("Request received:\n", JSON.stringify(event));
            var physicalId = event.PhysicalResourceId;
            function success(data) {
              return response.send(event, context, response.SUCCESS, data, physicalId);
            }
            function failed(e) {
              return response.send(event, context, response.FAILED, e, physicalId);
            }
            // Call ec2.waitFor, continuing if not finished before Lambda function timeout.
            function wait(waiter) {
              console.log("Waiting: ", JSON.stringify(waiter));
              event.waiter = waiter;
              event.PhysicalResourceId = physicalId;
              var request = ec2.waitFor(waiter.state, waiter.params);
              setTimeout(()=>{
                request.abort();
                console.log("Timeout reached, continuing function. Params:\n", JSON.stringify(event));
                var lambda = new AWS.Lambda();
                lambda.invoke({
                  FunctionName: context.invokedFunctionArn,
                  InvocationType: 'Event',
                  Payload: JSON.stringify(event)
                }).promise().then((data)=>context.done()).catch((err)=>context.fail(err));
              }, context.getRemainingTimeInMillis() - 5000);
              return request.promise().catch((err)=>
                (err.code == 'RequestAbortedError') ?
                  new Promise(()=>context.done()) :
                  Promise.reject(err)
              );
            }
            var ec2 = new AWS.EC2(),
                instanceId = event.ResourceProperties.InstanceId;
            if (event.waiter) {
              wait(event.waiter).then((data)=>success({})).catch((err)=>failed(err));
            } else if (event.RequestType == 'Create' || event.RequestType == 'Update') {
              if (!instanceId) { failed('InstanceID required'); }
              ec2.waitFor('instanceStopped', {InstanceIds: [instanceId]}).promise()
              .then((data)=>
                ec2.createImage({
                  InstanceId: instanceId,
                  Name: event.RequestId
                }).promise()
              ).then((data)=>
                wait({
                  state: 'imageAvailable',
                  params: {ImageIds: [physicalId = data.ImageId]}
                })
              ).then((data)=>success({})).catch((err)=>failed(err));
            } else if (event.RequestType == 'Delete') {
              if (physicalId.indexOf('ami-') !== 0) { return success({});}
              ec2.describeImages({ImageIds: [physicalId]}).promise()
              .then((data)=>
                (data.Images.length == 0) ? success({}) :
                ec2.deregisterImage({ImageId: physicalId}).promise()
              ).then((data)=>
                ec2.describeSnapshots({Filters: [{
                  Name: 'description',
                  Values: ["*" + physicalId + "*"]
                }]}).promise()
              ).then((data)=>
                (data.Snapshots.length === 0) ? success({}) :
                ec2.deleteSnapshot({SnapshotId: data.Snapshots[0].SnapshotId}).promise()
              ).then((data)=>success({})).catch((err)=>failed(err));
            }
          };
      Runtime: nodejs4.3
      Timeout: 300
  LambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal: {Service: [lambda.amazonaws.com]}
          Action: ['sts:AssumeRole']
      Path: /
      ManagedPolicyArns:
      - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
      - arn:aws:iam::aws:policy/service-role/AWSLambdaRole
      Policies:
      - PolicyName: EC2Policy
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
            - Effect: Allow
              Action:
              - 'ec2:DescribeInstances'
              - 'ec2:DescribeImages'
              - 'ec2:CreateImage'
              - 'ec2:DeregisterImage'
              - 'ec2:DescribeSnapshots'
              - 'ec2:DeleteSnapshot'
              Resource: ['*']
Outputs:
  AMI:
    Value: !Ref AMI

答案 1 :(得分:5)

对于它的价值,这里是wjordan的AMIFunction定义in the original answer的Python变体。原始yaml中的所有其他资源保持不变:

AMIFunction:
  Type: AWS::Lambda::Function
  Properties:
    Handler: index.handler
    Role: !GetAtt LambdaExecutionRole.Arn
    Code:
      ZipFile: !Sub |
        import logging
        import cfnresponse
        import json
        import boto3
        from threading import Timer
        from botocore.exceptions import WaiterError

        logger = logging.getLogger()
        logger.setLevel(logging.INFO)

        def handler(event, context):

          ec2 = boto3.resource('ec2')
          physicalId = event['PhysicalResourceId'] if 'PhysicalResourceId' in event else None

          def success(data={}):
            cfnresponse.send(event, context, cfnresponse.SUCCESS, data, physicalId)

          def failed(e):
            cfnresponse.send(event, context, cfnresponse.FAILED, str(e), physicalId)

          logger.info('Request received: %s\n' % json.dumps(event))

          try:
            instanceId = event['ResourceProperties']['InstanceId']
            if (not instanceId):
              raise 'InstanceID required'

            if not 'RequestType' in event:
              success({'Data': 'Unhandled request type'})
              return

            if event['RequestType'] == 'Delete':
              if (not physicalId.startswith('ami-')):
                raise 'Unknown PhysicalId: %s' % physicalId

              ec2client = boto3.client('ec2')
              images = ec2client.describe_images(ImageIds=[physicalId])
              for image in images['Images']:
                ec2.Image(image['ImageId']).deregister()
                snapshots = ([bdm['Ebs']['SnapshotId'] 
                              for bdm in image['BlockDeviceMappings'] 
                              if 'Ebs' in bdm and 'SnapshotId' in bdm['Ebs']])
                for snapshot in snapshots:
                  ec2.Snapshot(snapshot).delete()

              success({'Data': 'OK'})
            elif event['RequestType'] in set(['Create', 'Update']):
              if not physicalId:  # AMI creation has not been requested yet
                instance = ec2.Instance(instanceId)
                instance.wait_until_stopped()

                image = instance.create_image(Name="Automatic from CloudFormation stack ${AWS::StackName}")

                physicalId = image.image_id
              else:
                logger.info('Continuing in awaiting image available: %s\n' % physicalId)

              ec2client = boto3.client('ec2')
              waiter = ec2client.get_waiter('image_available')

              try:
                waiter.wait(ImageIds=[physicalId], WaiterConfig={'Delay': 30, 'MaxAttempts': 6})
              except WaiterError as e:
                # Request the same event but set PhysicalResourceId so that the AMI is not created again
                event['PhysicalResourceId'] = physicalId
                logger.info('Timeout reached, continuing function: %s\n' % json.dumps(event))
                lambda_client = boto3.client('lambda')
                lambda_client.invoke(FunctionName=context.invoked_function_arn, 
                                      InvocationType='Event',
                                      Payload=json.dumps(event))
                return

              success({'Data': 'OK'})
            else:
              success({'Data': 'OK'})
          except Exception as e:
            failed(e)
    Runtime: python2.7
    Timeout: 300

答案 2 :(得分:2)

  1. 没有
  2. 我想是的。一旦堆栈,您可以使用“更新堆栈”操作。你需要提供初始堆栈的完整JSON模板+你在同一个文件中的更改(Changed AMI)我会首先在测试环境中运行它(不是生产),因为我不确定操作对于现有实例。
  3. 为什么不在最初的云形态之外创建AMI,然后在最终的云形态模板中使用AMI?

    另一种选择是编写一些自动化来创建两个云形成堆栈,一旦你创建的AMI完成,你就可以删除第一个。

答案 3 :(得分:1)

虽然@ wjdordan的解决方案适用于简单的用例,但更新用户数据不会更新AMI。

(免责声明:我是原作者)cloudformation-ami旨在允许您在CloudFormation中声明可以可靠地创建,更新和删除的AMI。使用cloudformation-ami您可以声明这样的自定义AMI:

MyAMI:
  Type: Custom::AMI
  Properties:
    ServiceToken: !ImportValue AMILambdaFunctionArn
    Image:
      Name: my-image
      Description: some description for the image
    TemplateInstance:
      ImageId: ami-467ca739
      IamInstanceProfile:
        Arn: arn:aws:iam::1234567890:instance-profile/MyProfile-ASDNSDLKJ
      UserData:
        Fn::Base64: !Sub |
          #!/bin/bash -x
          yum -y install mysql # provisioning example
          # Signal that the instance is ready
          INSTANCE_ID=`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`
          aws ec2 create-tags --resources $INSTANCE_ID --tags Key=UserDataFinished,Value=true --region ${AWS::Region}
      KeyName: my-key
      InstanceType: t2.nano
      SecurityGroupIds:
      - sg-d7bf78b0
      SubnetId: subnet-ba03aa91
      BlockDeviceMappings:
      - DeviceName: "/dev/xvda"
        Ebs:
          VolumeSize: '10'
          VolumeType: gp2
相关问题