如何在CodePipeline中处理多个环境？
我使用代码管道来部署我的基础架构，我希望能够在不同的环境（dev，staging，prod，...）中部署它。
我目前有一个buildspec.yml文件，其中包含一些＆＃34; pip install＆＃34;说明和＆＃34; aws cloudformation包＆＃34;命令。我还创建了2个管道，一个用于生产，另一个用于开发，指向github上的2个不同分支。我的问题是，因为在两个分支中文件包含类似的资源，例如我在S3存储桶上有名称冲突。
使用AWS CLI和cloudformation创建或更新堆栈时，您可以使用--parameters选项传递参数。我想在我创建的2条管道中做类似的事情。
解决此问题的最佳解决方案是什么？
最终目标是自动部署我们的基础架构。我们的基础设施由用户，KMS密钥，Lamdbas（在python中），组和桶组成。
我在教程后创建了两个管道：http://docs.aws.amazon.com/lambda/latest/dg/automating-deployment.html
第一个管道链接到包含代码的repo的主分支，第二个管道链接到分段分支。我的目标是使用第一个管道和登台环境中的登台分支，使用第二个管道在生产环境中自动部署主分支。
我的buildspec.yml文件看起来像：
version: 0.1
phases:
install:
commands:
- pip install requests -t .
- pip install simplejson -t .
- pip install Image -t .
- aws cloudformation package --template-file image_processing_sam.yml --s3-bucket package-bucket --output-template-file new_image_processing_sam.yml
artifacts:
type: zip
files:
- new_image_processing_sam.yml
image_processing_sam.yml文件如下所示：
AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: Create a thumbnail for an image uploaded to S3
Resources:
ThumbnailFunction:
Type: "AWS::Serverless::Function"
Properties:
Role: !GetAtt LambdaExecutionRole.Arn
Handler: create_thumbnail.handler
Runtime: python2.7
Timeout: 30
Description: "A function computing the thumbnail for an image."
LambdaSecretEncryptionKey:
Type: "AWS::KMS::Key"
Properties:
Description: "A key used to encrypt secrets used in the Lambda functions"
Enabled: True
EnableKeyRotation: False
KeyPolicy:
Version: "2012-10-17"
Id: "lambda-secret-encryption-key"
Statement:
-
Sid: "Allow administration of the key"
Effect: "Allow"
Principal:
AWS: "arn:aws:iam::xxxxxxxxxxxxx:role/cloudformation-lambda-execution-role"
Action:
- "kms:Create*"
- "kms:Describe*"
- "kms:Enable*"
- "kms:List*"
- "kms:Put*"
- "kms:Update*"
- "kms:Revoke*"
- "kms:Disable*"
- "kms:Get*"
- "kms:Delete*"
- "kms:ScheduleKeyDeletion"
- "kms:CancelKeyDeletion"
Resource: "*"
-
Sid: "Allow use of the key"
Effect: "Allow"
Principal:
AWS:
- !GetAtt LambdaExecutionRole.Arn
Action:
- "kms:Encrypt"
- "kms:Decrypt"
- "kms:ReEncrypt*"
- "kms:GenerateDataKey*"
- "kms:DescribeKey"
Resource: "*"
LambdaExecutionRole:
Type: "AWS::IAM::Role"
Properties:
RoleName: "LambdaExecutionRole"
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- "lambda.amazonaws.com"
Action:
- "sts:AssumeRole"
Policies:
-
PolicyName: LambdaKMS
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: "Allow"
Action:
- "kms:Decrypt"
Resource: "*"
-
Effect: "Allow"
Action:
- "lambda:InvokeFunction"
Resource: "*"
ManagedPolicyArns:
- "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
UserGroup:
Type: "AWS::IAM::Group"
LambdaTriggerUser:
Type: "AWS::IAM::User"
Properties:
UserName: "LambdaTriggerUser"
LambdaTriggerUserKeys:
Type: "AWS::IAM::AccessKey"
Properties:
UserName:
Ref: LambdaTriggerUser
Users:
Type: "AWS::IAM::UserToGroupAddition"
Properties:
GroupName:
Ref: UserGroup
Users:
- Ref: LambdaTriggerUser
Policies:
Type: "AWS::IAM::Policy"
Properties:
PolicyName: UserPolicy
PolicyDocument:
Statement:
-
Effect: "Allow"
Action:
- "lambda:InvokeFunction"
Resource:
- !GetAtt DispatcherFunction.Arn
Groups:
- Ref: UserGroup
PackageBucket:
Type: "AWS::S3::Bucket"
Properties:
BucketName: "package-bucket"
VersioningConfiguration:
Status: "Enabled"
Outputs:
LambdaTriggerUserAccessKey:
Value:
Ref: "LambdaTriggerUserKeys"
Description: "AWSAccessKeyId of LambdaTriggerUser"
LambdaTriggerUserSecretKey:
Value: !GetAtt LambdaTriggerUserKeys.SecretAccessKey
Description: "AWSSecretKey of LambdaTriggerUser"
我在两个管道中添加了一个部署操作，以执行测试版操作期间计算的更改集。
第一个管道像魅力一样工作，并做我希望它做的一切。每次我在主分支中推送代码时，都会部署它。
我面临的问题是，当我在登台分支中推送代码时，一切都在管道中工作，直到达到部署操作。部署操作尝试创建一个新堆栈，但由于它完全相同的buildspec.yml和处理的image_processing_sam.yml，我达到名称冲突，如下所示。
package-bucket already exists in stack arn:aws:cloudformation:eu-west-1:xxxxxxxxxxxx:stack/master/xxxxxx-xxxx-xxx-xxxx
LambdaTriggerUser already exists in stack arn:aws:cloudformation:eu-west-1:xxxxxxxxxxxx:stack/master/xxxxxx-xxxx-xxx-xxxx
LambdaExecutionRole already exists in stack arn:aws:cloudformation:eu-west-1:xxxxxxxxxxxx:stack/master/xxxxxx-xxxx-xxx-xxxx
...
有没有办法参数化buildspec.yml，以便能够为image_processing_sam.yml中的资源名称添加后缀？任何其他想法都是受欢迎的。
祝你好运。
3 个答案:
答案 0 :(得分：6)
模板配置文件通过如下参数文件应用于CodePipeline中的CloudFormation：
{
"Parameters" : {
"DBName" : "TestWordPressDB",
"DBPassword" : "TestDBRootPassword",
"DBRootPassword" : "TestDBRootPassword",
"DBUser" : "TestDBuser",
"KeyName" : "TestEC2KeyName"
}
}
将这些文件放在您的仓库的根目录中，并且可以至少以两种方式引用它们。
在您的CodePipeline CloudFormation中：
Configuration:
ActionMode: REPLACE_ON_FAILURE
RoleArn: !GetAtt [CFNRole, Arn]
StackName: !Ref TestStackName
TemplateConfiguration: !Sub "TemplateSource::${TestStackConfig}"
TemplatePath: !Sub "TemplateSource::${TemplateFileName}"
或者在模板配置字段的控制台中：
值得注意的是，配置文件格式与使用
的cli的CloudFormation不同-- parameters
- 参数使用以下格式：
[
{
"ParameterKey": "team",
"ParameterValue": "AD-Student Life Applications"
},
{
"ParameterKey": "env",
"ParameterValue": "dev"
},
{
"ParameterKey": "dataSensitivity",
"ParameterValue": "public"
},
{
"ParameterKey": "app",
"ParameterValue": "events-list-test"
}
]
CodePipeline Cloudformation模板配置文件使用以下格式：
{
"Parameters" : {
"DBName" : "TestWordPressDB",
"DBPassword" : "TestDBRootPassword",
"DBRootPassword" : "TestDBRootPassword",
"DBUser" : "TestDBuser",
"KeyName" : "TestEC2KeyName"
}
}
答案 1 :(得分：3)
检查Eric Nord的答案。这是你要找的那个。
我在AWS论坛上也提出了问题here。
以下是AWS提供的解决方案：
您好，
如果您的目标是为暂存和主控具有不同的存储桶名称，则另一个选项是使用CloudFormation参数。
如果编辑操作时编辑现有管道，则可以展开“高级”面板并输入参数覆盖，以便为每个阶段指定不同的存储桶前缀。您还可以在工件中将参数作为单独的.json文件输入。
以下是完整的测试和制作堆栈配置：http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/continuous-delivery-codepipeline-basic-walkthrough.html
- 添。
您一定要遵循提供的文档。这是我在下面提出的解决方案。
这是我自己的解决方案，我不满意。
我已经添加了一个在构建时运行的脚本，并根据构建项目的CodeBuild代理的ARN修改了我的模板。
我添加了“BRANCH_NAME”，可能会发生命名冲突。 image_processing_sam.yml现在是：
AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: Create a thumbnail for an image uploaded to S3
Resources:
ThumbnailFunction:
Type: "AWS::Serverless::Function"
Properties:
Role: !GetAtt LambdaExecutionRole.Arn
Handler: create_thumbnail.handler
Runtime: python2.7
Timeout: 30
Description: "A function computing the thumbnail for an image."
LambdaSecretEncryptionKey:
Type: "AWS::KMS::Key"
Properties:
Description: "A key used to encrypt secrets used in the Lambda functions"
Enabled: True
EnableKeyRotation: False
KeyPolicy:
Version: "2012-10-17"
Id: "lambda-secret-encryption-keyBRANCH_NAME"
Statement:
-
Sid: "Allow administration of the key"
Effect: "Allow"
Principal:
AWS: "arn:aws:iam::xxxxxxxxxxxxx:role/cloudformation-lambda-execution-role"
Action:
- "kms:Create*"
- "kms:Describe*"
- "kms:Enable*"
- "kms:List*"
- "kms:Put*"
- "kms:Update*"
- "kms:Revoke*"
- "kms:Disable*"
- "kms:Get*"
- "kms:Delete*"
- "kms:ScheduleKeyDeletion"
- "kms:CancelKeyDeletion"
Resource: "*"
-
Sid: "Allow use of the key"
Effect: "Allow"
Principal:
AWS:
- !GetAtt LambdaExecutionRole.Arn
Action:
- "kms:Encrypt"
- "kms:Decrypt"
- "kms:ReEncrypt*"
- "kms:GenerateDataKey*"
- "kms:DescribeKey"
Resource: "*"
LambdaExecutionRole:
Type: "AWS::IAM::Role"
Properties:
RoleName: "LambdaExecutionRoleBRANCH_NAME"
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- "lambda.amazonaws.com"
Action:
- "sts:AssumeRole"
Policies:
-
PolicyName: LambdaKMSBRANCH_NAME
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: "Allow"
Action:
- "kms:Decrypt"
Resource: "*"
-
Effect: "Allow"
Action:
- "lambda:InvokeFunction"
Resource: "*"
ManagedPolicyArns:
- "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
UserGroup:
Type: "AWS::IAM::Group"
LambdaTriggerUser:
Type: "AWS::IAM::User"
Properties:
UserName: "LambdaTriggerUserBRANCH_NAME"
LambdaTriggerUserKeys:
Type: "AWS::IAM::AccessKey"
Properties:
UserName:
Ref: LambdaTriggerUser
Users:
Type: "AWS::IAM::UserToGroupAddition"
Properties:
GroupName:
Ref: UserGroup
Users:
- Ref: LambdaTriggerUser
Policies:
Type: "AWS::IAM::Policy"
Properties:
PolicyName: UserPolicyBRANCH_NAME
PolicyDocument:
Statement:
-
Effect: "Allow"
Action:
- "lambda:InvokeFunction"
Resource:
- !GetAtt DispatcherFunction.Arn
Groups:
- Ref: UserGroup
PackageBucket:
Type: "AWS::S3::Bucket"
Properties:
BucketName: "package-bucketBRANCH_NAME"
VersioningConfiguration:
Status: "Enabled"
Outputs:
LambdaTriggerUserAccessKey:
Value:
Ref: "LambdaTriggerUserKeys"
Description: "AWSAccessKeyId of LambdaTriggerUser"
LambdaTriggerUserSecretKey:
Value: !GetAtt LambdaTriggerUserKeys.SecretAccessKey
Description: "AWSSecretKey of LambdaTriggerUser"
script.sh替换模板中的“BRANCH_NAME”：
#!/bin/bash
echo $CODEBUILD_AGENT_ENV_CODEBUILD_BUILD_ARN
if [[ "$CODEBUILD_AGENT_ENV_CODEBUILD_BUILD_ARN" == *"master"* ]]; then
sed "s/BRANCH_NAME//g" image_processing_sam.yml > generated_image_processing_sam.yml;
fi
if [[ "$CODEBUILD_AGENT_ENV_CODEBUILD_BUILD_ARN" == *"staging"* ]]; then
sed "s/BRANCH_NAME/staging/g" image_processing_sam.yml > generated_image_processing_sam.yml;
fi
buildspec.yml现在是：
version: 0.1
phases:
install:
commands:
# Install required module for python
- pip install requests -t .
- pip install simplejson -t .
- pip install Image -t .
- bash ./script.sh
# To be able to see any issue in the generated template
- cat generated_image_processing_sam.yml
# Package the generated cloudformation template in order to deploy
- aws cloudformation package --template-file generated_image_processing_sam.yml --s3-bucket piximate-package-bucket --output-template-file new_image_processing_sam.yml
artifacts:
type: zip
files:
- new_image_processing_sam.yml
我希望它可以以某种方式帮助你。如果有人能提供任何有用的改进或文件，我会很高兴。
答案 2 :(得分：0)
我遇到了同样的挑战，最终根据资源库的不同，以两种不同的方式编写了分步教程。如果您需要更多详细信息，这是我写的帖子的链接，但是TLDR涵盖了很多内容。
How to Track Multiple Git Branches in AWS CodePipeline
[TL; DR]
GitHub或Bitbucket：使用Webhook创建一个CodeBuild项目，并使用S3作为源。
CodeCommit：创建一个Lambda触发器，该触发器为每个分支创建一个新管道。
相关问题
最新问题
- 我写了这段代码，但我无法理解我的错误
- 我无法从一个代码实例的列表中删除 None 值，但我可以在另一个实例中。为什么它适用于一个细分市场而不适用于另一个细分市场？
- 是否有可能使 loadstring 不可能等于打印？卢阿
- java中的random.expovariate()
- Appscript 通过会议在 Google 日历中发送电子邮件和创建活动
- 为什么我的 Onclick 箭头功能在 React 中不起作用？
- 在此代码中是否有使用“this”的替代方法？
- 在 SQL Server 和 PostgreSQL 上查询，我如何从第一个表获得第二个表的可视化
- 每千个数字得到
- 更新了城市边界 KML 文件的来源？