Question

我正在尝试在AWS Lambda中创建一个Python 3.7函数。但是在测试我的功能时遇到错误-

{
  "errorMessage": "Unable to import module 'lambda_function': No module named 's3fs'",
  "errorType": "Runtime.ImportModuleError"
}

我知道，默认情况下，Python环境没有导入s3fs模块。因此，我不得不在本地计算机上安装s3fs和其他一些软件包，并将其压缩。对于压缩，我使用了以下命令。

Compress-Archive -Path dateutil, docutils, jmespath, s3fs, s3transfer, six.py ` -DestinationPath ..\..\..\pinpoint-importer.zip ;

（我遵循了以下文档-https://docs.aws.amazon.com/pinpoint/latest/developerguide/tutorials-importing-data-create-python-package.html）。

然后将.zip文件上传到Lambda中，并交叉检查处理程序-“ lambda_function.lambda_handler”。

请在下面找到代码。

import os
import boto3
import s3fs
from botocore.exceptions import ClientError

input_archive_folder = "input_archive"
to_process_folder = "to_process"
file_row_limit = 50
file_delimiter = ','

# S3 bucket info
s3 = s3fs.S3FileSystem(anon=False)

def lambda_handler(event, context):
    print("Received event: \n" + str(event))
    for record in event['Records']:
        # Assign some variables that make it easier to work with the data in the 
        # event record.
        bucket = record['s3']['bucket']['name']
        key = record['s3']['object']['key']
        input_file = os.path.join(bucket,key)
        archive_path = os.path.join(bucket,input_archive_folder,os.path.basename(key))
        folder =  os.path.split(key)[0]
        s3_url = os.path.join(bucket,folder)
        output_file_template = os.path.splitext(os.path.basename(key))[0] + "__part"
        output_path = os.path.join(bucket,to_process_folder)

        # Set a variable that contains the number of files that this Lambda 
        # function creates after it runs.
        num_files = file_count(s3.open(input_file, 'r'), file_delimiter, file_row_limit)

        # Split the input file into several files, each with 50 rows.
        split(s3.open(input_file, 'r'), file_delimiter, file_row_limit, output_file_template, output_path, True, num_files)

        # Send the unchanged input file to an archive folder.
        archive(input_file,archive_path)

# Determine the number of files that this Lambda function will create.
def file_count(file_handler, delimiter, row_limit):
    import csv 
    reader = csv.reader(file_handler, delimiter=delimiter)
    # Figure out the number of files this function will generate.
    row_count = sum(1 for row in reader) - 1
    # If there's a remainder, always round up.
    file_count = int(row_count // row_limit) + (row_count % row_limit > 0)
    return file_count

# Split the input into several smaller files.
def split(filehandler, delimiter, row_limit, output_name_template, output_path, keep_headers, num_files):
    import csv 
    reader = csv.reader(filehandler, delimiter=delimiter)

    current_piece = 1
    current_out_path = os.path.join(
         output_path,
         output_name_template + str(current_piece) + "__of" + str(num_files) + ".csv"
    )
    current_out_writer = csv.writer(s3.open(current_out_path, 'w'), delimiter=delimiter)
    current_limit = row_limit
    if keep_headers:
        headers = next(reader)
        current_out_writer.writerow(headers)
    for i, row in enumerate(reader):
        if i + 1 > current_limit:
            current_piece += 1
            current_limit = row_limit * current_piece
            current_out_path = os.path.join(
               output_path,
               output_name_template + str(current_piece) + "__of" + str(num_files) + ".csv"
            )
            current_out_writer = csv.writer(s3.open(current_out_path, 'w'), delimiter=delimiter)
            if keep_headers:
                current_out_writer.writerow(headers)
        current_out_writer.writerow(row)

# Move the original input file into an archive folder.
def archive(input_file, archive_path):
    s3.copy_basic(input_file,archive_path)
    print("Moved " + input_file + " to " + archive_path)
    s3.rm(input_file)

我遵循了本文档-https://docs.aws.amazon.com/pinpoint/latest/developerguide/tutorials-importing-data-lambda-function-input-split.html

屏幕截图-

如果我缺少什么，请告诉我。谢谢。

Answer 1

好吧，我找到了解决此问题的方法。尽管问题的根本原因尚不明确。

我在本地系统中粘贴了代码（可以在AWS文档中找到），并压缩了整个目录，而不是指定的库（在AWS文档中指定-步骤8-https://docs.aws.amazon.com/pinpoint/latest/developerguide/tutorials-importing-data-create-python-package.html），以及已编辑的内容文件-lambda_function.py。
然后在S3中上传了zip，并在Lambda函数代码部分中拉出了zip。
而且有效！

我会要求观众测试该文档https://docs.aws.amazon.com/pinpoint/latest/developerguide/tutorials-importing-data-create-python-package.html，并让我知道我是否错过了某些内容，或者如果可能的话，需要对文档进行更正。非常感谢！

Answer 2

如果将错误消息中显示的所有文件添加到zip文件中，然后手动上传一次，则可以使用。我认为需要通知AWS，因为他们的指南没有提及有关此类错误的任何信息。

无法将库导入AWS Lambda

2 个答案: