通过无服务器部署时,为什么docker无法找到我自己的软件包?

时间:2018-09-29 15:49:17

标签: docker aws-lambda serverless-framework

我想将python软件包部署到Amazon并通过lambda使其可用。为此,我正在尝试无服务器。

当我尝试部署程序包时,出现以下错误消息:

SLS_DEBUG=* serverless deploy --stage dev --aws-profile default
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command create
Serverless: Load command install
Serverless: Load command package
Serverless: Load command deploy
Serverless: Load command deploy:function
Serverless: Load command deploy:list
Serverless: Load command deploy:list:functions
Serverless: Load command invoke
Serverless: Load command invoke:local
Serverless: Load command info
Serverless: Load command logs
Serverless: Load command login
Serverless: Load command logout
Serverless: Load command metrics
Serverless: Load command print
Serverless: Load command remove
Serverless: Load command rollback
Serverless: Load command rollback:function
Serverless: Load command slstats
Serverless: Load command plugin
Serverless: Load command plugin
Serverless: Load command plugin:install
Serverless: Load command plugin
Serverless: Load command plugin:uninstall
Serverless: Load command plugin
Serverless: Load command plugin:list
Serverless: Load command plugin
Serverless: Load command plugin:search
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command rollback
Serverless: Load command rollback:function
Serverless: Load command requirements
Serverless: Load command requirements:clean
Serverless: Load command requirements:install
Serverless: Load command requirements:cleanCache
Serverless: Invoke deploy
Serverless: Invoke package
Serverless: Invoke aws:common:validate
Serverless: Invoke aws:common:cleanupTempDir
Serverless: Generated requirements from /home/ola/projects/lambda/tagdoc/requirements.txt in /home/ola/projects/lambda/tagdoc/.serverless/requirements.txt...
Serverless: Installing requirements from /home/ola/projects/lambda/tagdoc/.serverless/requirements/requirements.txt ...
Serverless: Docker Image: lambci/lambda:build-python3.6
Requirement 'pkgg-0.1.0.tar.gz' looks like a filename, but the file does not exist
Processing ./pkgg-0.1.0.tar.gz
Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/var/task/pkgg-0.1.0.tar.gz'


  Error --------------------------------------------------

  null

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Stack Trace --------------------------------------------

Error: null
    at installRequirements (/home/ola/projects/lambda/tagdoc/node_modules/serverless-python-requirements/lib/pip.js:262:11)
    at installRequirementsIfNeeded (/home/ola/projects/lambda/tagdoc/node_modules/serverless-python-requirements/lib/pip.js:448:3)
    at ServerlessPythonRequirements.installAllRequirements (/home/ola/projects/lambda/tagdoc/node_modules/serverless-python-requirements/lib/pip.js:527:29)
From previous event:
    at PluginManager.invoke (/usr/lib/node_modules/serverless/lib/classes/PluginManager.js:390:22)
    at PluginManager.spawn (/usr/lib/node_modules/serverless/lib/classes/PluginManager.js:408:17)
    at Deploy.BbPromise.bind.then.then (/usr/lib/node_modules/serverless/lib/plugins/deploy/deploy.js:123:50)
From previous event:
    at Object.before:deploy:deploy [as hook] (/usr/lib/node_modules/serverless/lib/plugins/deploy/deploy.js:113:10)
    at BbPromise.reduce (/usr/lib/node_modules/serverless/lib/classes/PluginManager.js:390:55)
From previous event:
    at PluginManager.invoke (/usr/lib/node_modules/serverless/lib/classes/PluginManager.js:390:22)
    at PluginManager.run (/usr/lib/node_modules/serverless/lib/classes/PluginManager.js:421:17)
    at variables.populateService.then.then (/usr/lib/node_modules/serverless/lib/Serverless.js:157:33)
    at runCallback (timers.js:810:20)
    at tryOnImmediate (timers.js:768:5)
    at processImmediate [as _immediateCallback] (timers.js:745:5)
From previous event:
    at Serverless.run (/usr/lib/node_modules/serverless/lib/Serverless.js:144:8)
    at serverless.init.then (/usr/lib/node_modules/serverless/bin/serverless:43:50)
    at <anonymous>

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information -----------------------------
     OS:                     linux
     Node Version:           8.15.0
     Serverless Version:     1.30.1

我的serverless.yml文件如下:

# Welcome to Serverless!
#
# This file is the main config file for your service.
# It's very minimal at this point and uses default values.
# You can always add more config options for more control.
# We've included some commented out config examples here.
# Just uncomment any of them to get that config option.
#
# For full config options, check the docs:
#    docs.serverless.com
#
# Happy Coding!

service: tagdoc # NOTE: update this with your service name

# You can pin your service to only deploy with a specific Serverless version
# Check out our docs for more details
# frameworkVersion: "=X.X.X"

provider:
  name: aws
  runtime: python3.6
  region: eu-central-1
# you can overwrite defaults here
#  stage: dev

plugins:
  - serverless-python-requirements

# You need to have Docker installed to be able to set dockerizePip:
# true or dockerizePip: non-linux. Alternatively, you can set
# dockerizePip: false, and it will not use Docker packaging. But,
# Docker packaging is essential if you need to build native packages
# that are part of your dependencies like Psycopg2, NumPy, Pandas, etc
custom:
  pythonRequirements:
    dockerizePip: true

# you can define service wide environment variables here
#  environment:
#    variable1: value1

# you can add packaging information here
package:
  include:
    - ./nltk_data/*

functions:
  ttxt:
    handler: handler.ttxt
    events:
      - http:
          path: /ttxt
          method: get
          integration: lambda
          request:
            template:
              application/json: '{ "txt" : "$input.params(''txt'')" }'
          response:
            headers:
  turl:
    handler: handler.turl
    events:
      - http:
          path: /turl
          method: get
          integration: lambda
          request:
            template:
              application/json: '{ "url" : "$input.params(''url'')" }'
          response:
            headers:
              Content-Type: "'application/json'"

最重要的是,该目录包括所有必需的文件,尤其是内置的python包:

tree -L 2 lambda/
lambda/
└── tagdoc
    ├── pkgg-0.1.0.tar.gz
    ├── handler.py
    ├── nltk_data
    ├── node_modules
    ├── package.json
    ├── package-lock.json
    ├── README.md
    ├── requirements.txt
    ├── serverless.yml
    └── srv

4 directories, 8 files

我当前正在运行Debian 9.5。我还与一个通过Github运行相同操作系统的朋友共享了代码,看来他可以部署该软件包。因此,它看起来与我在本地的无服务器/ docker安装有关。但是我不知道如何解决这个问题。任何帮助将不胜感激。

requirements.txt仅包含一行:

cat requirements.txt 
pkgg-0.1.0.tar.gz

现在,程序包本身具有一定的依赖性。通过仅包含该程序包,它就可以从该程序包的setup.py文件中获取依赖项。

编辑

根据评论之一的要求。这是我的.gitignore

#Compiled source#
#################
*.pyc

# no data files #
################
*.csv

#Log files#
#################
*.log

#swap files#
###############
*.swp
*.*~
\#*\#
.\#*

#django migration directory#
############################
venv*
srv*
theme
collectedstatic
stunnel

# Elastic Beanstalk Files
.elasticbeanstalk/*
!.elasticbeanstalk/*.cfg.yml
!.elasticbeanstalk/*.global.yml

#Serverless package
.serverless

这就是我的.serverless的样子。所有requirements.txt的内容与上述内容完全相同。

tree -L 3 .serverless/
.serverless/
├── cloudformation-template-create-stack.json
├── requirements
│   └── requirements.txt
└── requirements.txt

1 directory, 3 files

4 个答案:

答案 0 :(得分:4)

问题似乎在serverless-python-requirements之内。就像在4.1.1版本中看起来一样,它可以完美地工作,而在4.2.5中则不能。我在github上提出了一个问题

答案 1 :(得分:1)

dockerizePip: true => pip将在容器中运行,该容器默认情况下不与主机OS共享文件系统,但可以从Internet下载python软件包。看来插件serverless-python-requirements无法将您的本地文件正确地装载到容器中,因此pip无法看到您的本地包(pkgg-0.1.0.tar.gz)。

恕我直言,最好的解决方案是不要使用requirements.txt中的本地文件。

交叉引用:https://github.com/UnitedIncome/serverless-python-requirements/issues/258

答案 2 :(得分:0)

项目lambci/lambda将自己描述为Images that (very closely) mimic the live AWS Lambda environment,如果要确保环境100%兼容,请在EC2上启动一个小的t2.micro实例,通过SSH连接并在那里设置项目,在您的Shell上运行pip install -r requirements.txt,下载所有库并处置该实例。这就是Lambda下一次为其提供the environment来运行代码时将要做的事情。

答案 3 :(得分:0)

考虑按照无服务器blog post

所述在Docker容器中构建