用于AWS Lambda的压缩包(带有numpy)不起作用

时间:2018-02-22 16:01:02

标签: python numpy amazon-s3 zip aws-lambda

尝试从位于S3的Zip文件中运行AWS Lambda(Python 3.6)中的函数。

遵循各个地点的说明(AWS文档,其他博客等)。但是,在加载numpy的时候,我似乎在某处出错。

有人可以帮忙吗?

以下是我上传到S3的Zip文件 - ZIP file

此外,当我测试zip -

时,错误日志会返回以下内容
START RequestId: b004d192-17e7-11e8-8323-bb9e005088cc Version: $LATEST
Unable to import module 'trial22': 
Importing the multiarray numpy extension module failed.  Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try `git clean -xdf` (removes all files not under version control).  Otherwise reinstall numpy.

Original error was: cannot import name 'multiarray'


END RequestId: b004d192-17e7-11e8-8323-bb9e005088cc
REPORT RequestId: b004d192-17e7-11e8-8323-bb9e005088cc  Duration: 0.84 ms   Billed Duration: 100 ms     Memory Size: 128 MB Max Memory Used: 31 MB

在尝试卸载(& then install)numpy时,virtualenv会尝试删除全局包,而不是创建它的特定目录中的包。

Not uninstalling numpy at /Users/vidyut/projects/aws_lambda_pkgs/trial22, outside environment /Users/vidyut/projects/aws_lambda_pkgs/trial21/bin/..

我知道我在某个地方犯了一些愚蠢的错误,但根本无法弄清楚在哪里和哪里 - 有人可以帮忙解释一下吗?

1 个答案:

答案 0 :(得分:0)

以下是我收到的AWS支持回复:

I seem to have managed to get the function working. Please see below the steps which I have followed. I initially tried adding the dependancies using the virtualenv method but got the same errors as your were getting. Then I tried using the method described at the below link using an EC2 instance launched with the same execution environment as Lambda and pip. Please can you give this a try.

https://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html

Create an EC2 instance using the AMI mentioned in the below document - AMI name: amzn-ami-hvm-2017.03.1.20170812-x86_64-gp2.

Go to EC2 console, Launch Instance and click Community AMI’s, then paste the AMI name - amzn-ami-hvm-2017.03.1.20170812-x86_64-gp2 - into the search box and hit enter. Select the AMI and continue with the EC2 instance setup.

https://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html

Once the instance has been created, ssh into the instance and follow the steps below.

1. Create working directory
mkdir lambda
cd lambda/

2. Install required packages and dependancies
sudo yum -y groupinstall "Development Tools"
sudo yum install python27-devel python27-pip gcc
sudo pip install numpy -t ~/lambda/
sudo pip install sqlalchemy -t ~/lambda/
sudo pip install pandas -t ~/lambda/


3. SCP the Lambda function file to ~/lambda directory on EC2 instance or create a lambda_function.py file in the ~/lambda directory

e.g.

scp -i Key.pem lambda_function.py ec2-user@ec2-xx-xx-xxx-xxx.amazonaws.com:~/lambda/lambda.zip

or

nano/vi lambda_function.py and paste the code into the terminal
from the ~/lambda directory run - zip -r "lambda.zip" *

4. Upload the lambda.zip file to S3 using the AWS CLI or SCP the file to you local machine and upload to S3 from the S3 console.

e.g.

Copying a local file to S3
The following cp command copies a single file to a specified bucket and key:
aws s3 cp test.txt s3://mybucket/test2.txt
https://docs.aws.amazon.com/de_de/cli/latest/reference/s3/cp.html

or

scp -i Key.pem ec2-user@ec2-xx-xx-xxx-xxx.amazonaws.com:~/lambda/lambda.zip ~/lambda.zip

5. Add file to Lambda from S3 and choose Python2.7 as runtime.

6. Test function. No error returned, see output below.

START RequestId: c8e02a09-1a06-11e8-8ac0-17072432f975 Version: $LATEST
[[6.92879737e-310 6.92879737e-310]
 [8.86188553e-317 8.86183810e-317]]
END RequestId: c8e02a09-1a06-11e8-8ac0-17072432f975
REPORT RequestId: c8e02a09-1a06-11e8-8ac0-17072432f975  Duration: 0.85 ms   Billed Duration: 100 ms     Memory Size: 128 MB Max Memory Used: 26 MB