如何从Jenkins将Python应用程序部署到Amazon Elastic Beanstalk?

时间:2014-12-19 18:04:37

标签: python amazon-web-services jenkins amazon-elastic-beanstalk

我正尝试从Jenkins作业以编程方式部署到Amazon Elastic Beanstalk。在我的开发机器上,这很简单:

eb deploy $(AWS_ELASTIC_BEANSTALK_ENVIRONMENT)

在Jenkins上,它应该像将以下内容配置为构建命令一样简单:

virtualenv env && source env/bin/activate && pip install awsebcli
mkdir -p .elasticbeanstalk
cat << EOF > .elasticbeanstalk/config.yml 
branch-defaults:
  master:
    environment:  myenv
global:
  application_name:  myapp
  default_ec2_keyname: null
  default_platform: 64bit Amazon Linux 2014.09 v1.0.9 running Python 2.7
  default_region: us-west-2
  profile: eb-cli
  sc: git
EOF
eb deploy myenv

然而,这失败了以下跟踪:

[EnvInject] - Loading node environment variables.
Building remotely on standard Amazon Linux 2014.09 AMI (i-d39710df) (x) in workspace /media/ephemeral0/jenkins/workspace/My_Job
Fetching changes from the remote Git repository
Fetching upstream changes from git@github.com:myapp.git
Checking out Revision be45db94111f9ab49fe8031eb583307d2fb9921c (origin/master)
[My_Job] $ /bin/sh -xe /tmp/hudson8633484437962332339.sh
+ virtualenv env
New python executable in env/bin/python2.7
Not overwriting existing python script env/bin/python (you must use env/bin/python2.7)
Installing Setuptools..................................................................................................................done.
Installing Pip....................................................................................................................................done.
+ source env/bin/activate
++ deactivate nondestructive
++ unset pydoc
++ '[' -n '' ']'
++ '[' -n '' ']'
++ '[' -n /bin/sh -o -n '' ']'
++ hash -r
++ '[' -n '' ']'
++ unset VIRTUAL_ENV
++ '[' '!' nondestructive = nondestructive ']'
++ VIRTUAL_ENV=/media/ephemeral0/jenkins/workspace/My_Job/env
++ export VIRTUAL_ENV
++ _OLD_VIRTUAL_PATH=/usr/local/bin:/bin:/usr/bin:/opt/aws/bin
++ PATH=/media/ephemeral0/jenkins/workspace/My_Job/env/bin:/usr/local/bin:/bin:/usr/bin:/opt/aws/bin
++ export PATH
++ '[' -n '' ']'
++ '[' -z '' ']'
++ _OLD_VIRTUAL_PS1=
++ '[' x '!=' x ']'
+++ basename /media/ephemeral0/jenkins/workspace/My_Job/env
++ '[' env = __ ']'
+++ basename /media/ephemeral0/jenkins/workspace/My_Job/env
++ PS1='(env)'
++ export PS1
++ alias 'pydoc=python -m pydoc'
++ '[' -n /bin/sh -o -n '' ']'
++ hash -r
+ pip install awsebcli
Requirement already satisfied (use --upgrade to upgrade): awsebcli in ./env/lib/python2.7/site-packages
Downloading/unpacking setuptools>=7.0 (from awsebcli)
  Running setup.py egg_info for package setuptools

Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.11 in ./env/lib/python2.7/site-packages (from awsebcli)
Requirement already satisfied (use --upgrade to upgrade): six==1.8.0 in ./env/lib/python2.7/site-packages (from awsebcli)
Requirement already satisfied (use --upgrade to upgrade): cement==2.4 in ./env/lib/python2.7/site-packages (from awsebcli)
Requirement already satisfied (use --upgrade to upgrade): python-dateutil>=2.2 in ./env/lib/python2.7/site-packages (from awsebcli)
Requirement already satisfied (use --upgrade to upgrade): jmespath>=0.4.1 in ./env/lib/python2.7/site-packages (from awsebcli)
Installing collected packages: setuptools
  Found existing installation: setuptools 0.9.7
    Uninstalling setuptools:
      Successfully uninstalled setuptools
  Running setup.py install for setuptools

    Installing easy_install script to /media/ephemeral0/jenkins/workspace/My_Job/env/bin
    Installing easy_install-2.7 script to /media/ephemeral0/jenkins/workspace/My_Job/env/bin
Successfully installed setuptools
Cleaning up...
+ mkdir -p .elasticbeanstalk
+ cat
+ cat .elasticbeanstalk/config.yml
branch-defaults:
  master:
    environment: myenv
global:
  application_name: myapp
  default_ec2_keyname: null
  default_platform: 64bit Amazon Linux 2014.09 v1.0.9 running Python 2.7
  default_region: us-west-2
  profile: eb-cli
  sc: git
+ eb deploy myenv
ERROR: The config profile (eb-cli) could not be found
Build step 'Execute shell' marked build as failure
Finished: FAILURE

目前还不清楚为什么会发生这种情况,因为当我在项目的本地副本上运行上述内容时,它运行正常。

错误消息似乎没有多大帮助。目前还不清楚为什么在Jenkins机器上找不到eb-cli。

再次总结一下我的问题:如何从Jenkins部署到Amazon Elastic Beanstalk?上述方法是否正确,但细节有误?或者是否有一些更简单的方法?

3 个答案:

答案 0 :(得分:12)

要更正config profile (eb-cli) could not be found错误,请将您用于部署到EB的凭据删除到jenkins计算机上的jenkins用户的~/.aws/config。如果您在本地计算机上构建了部署,则应该能够直接从本地~/.aws/config提取文件。它看起来像这样:

[profile eb-cli]
aws_access_key_id = (for your IAM user)
aws_secret_access_key = (for your IAM user)

答案 1 :(得分:3)

我通过ssh解析了Jenkins机器,运行eb init,然后将生成的.elasticbeanstalk/config.yml与我上面使用的here-doc中的那个进行比较。这两者是不同的,因为我的开发机器上的安全配置文件与Jenkins机器不同。

我们可以重写此脚本,以便对不同的config.yaml文件更加健壮,如下所示:

virtualenv env && source env/bin/activate && pip install awsebcli
echo "1" | eb init myapp --region us-west-2 && eb use myenv && eb deploy myenv

注意,我们使用echo "1" | eb init myapp --region us-west-2来选择默认环境,因为eb init不将环境视为位置参数,然后使用eb use myenv来选择我们想要的环境。

答案 2 :(得分:0)

我们突然遇到了这个问题,这可能或者可能不相关,但是我想让那些可能再遇到这个问题的人知道这个问题。 eb-cli似乎略有改变,不允许全局设置凭证。

我们的JenkinsFile看起来像这样

 withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', 
 credentialsId: 'iam-creds']]) {
       sh "pip install awsebcli --upgrade --user"
       sh "~/.local/bin/eb use my-application"
       sh "~/.local/bin/eb deploy --verbose"
     }

我们的config.yml看起来像这样

 global:
   application_name: my-application
   default_ec2_keyname: application-keyname
   ...
   profile: eb-cli
   sc: git
   workspace_type: Application

删除profile密钥解决了问题...但是,这不允许从本地计算机部署(除非有办法使用全局变量)