挂在等待s3.listObjects

时间:2019-07-16 16:30:47

标签: javascript node.js aws-lambda aws-sdk-nodejs

我有一个事件触发lambda代码,该代码尝试列出一个存储桶中的所有对象,然后将它们复制到另一个存储桶中。

使用事件调用处理程序的代码

const cfnHandlerWrapper = async (event, handler) => {
    let resultPromise;
    try {
        handler(event).then(function() {
            console.log(`Successfully copied over all artifacts`);
            resultPromise = sendSuccess(event);
        }).catch(function(error) {
            console.log(`Failed copying over all artifacts: ${error}`);
            throw error;
        });
    } catch (e) {
        console.log('ERROR RUNNING HANDLER:');
        console.log(e);
        resultPromise = sendFailure(e.message || 'Something went wrong', event);
    }
    return resultPromise;
};

处理复制的代码

const copyHandler = async (event) => {
    console.log('Running Agent artifacts copy handler');

    const {RequestType, ResourceProperties} = event;
    const {DestBucket, SrcBucket, AdditionalArtifactsPath} = ResourceProperties;

    if (RequestType === 'Create' || RequestType === 'Update') {
        const srcLocation = [SrcBucket, AdditionalArtifactsPath].join('/');
        const batsAdditionalArtifactsParams = {
            Bucket: SrcBucket,
            Delimiter: '',
            Prefix: `${AdditionalArtifactsPath}/`
        };

        let copyPromises = [];
        console.log("about to await s3.listObjects()...");

        let listObjectsResult;
        try {
            listObjectsResult = await s3.listObjects(batsAdditionalArtifactsParams).promise();
        } catch(err) {
            console.log(`Failed to list objects for ${srcLocation}, err: ${err}`);
            throw err;
        }
        console.log(`finished waiting for listObjects. listObjectsResult: ${listObjectsResult}`);
        console.log(`Successfully listed objects in ${srcLocation}. Attempting to copy all artifacts to ${DestBucket}`);
        listObjectsResult.Contents.forEach((object) => {
            console.log(`parsing object ${object}`);
            let keyParts = object.Key.split('/');
            let fileKey = keyParts.pop();
            let destKey = fileKey.includes(agentBootstrapScript) ? `${agentBootstrapScriptsFolder}/${agentBootstrapScript}` : fileKey; // Adhere to bootstrap folder structure
            let copyParams = {
                Bucket: DestBucket,
                CopySource: `${srcLocation}/${fileKey}`,
                Key: destKey
            };
            console.log("pushing copy promise");
            copyPromises.push(s3.copyObject(copyParams).promise()
                .then(function(data) {
                    console.log(`Successfully copied ${fileKey} from ${srcLocation} to ${DestBucket}! data=${data}`);
                }).catch(function(err) {
                console.log(`Encountered error while copying ${fileKey} from ${srcLocation} to ${DestBucket}, error: ${err}`);
            }));
        });
        console.log("about to await Promise.all()....");
        await Promise.all(copyPromises);
    } else {
        console.log(`Received event type ${RequestType}, not copying anything`);
    }

    console.log("returning Promise.resolve()");
    return Promise.resolve();
};

此代码仅输出日志消息 about to await s3.listObjects()...,因此似乎正等待等待:

listObjectsResult = await s3.listObjects(batsAdditionalArtifactsParams).promise();

我要做的就是等待所有对象列出,承诺复制这些对象,然后使用Promise.all在从该处理程序返回之前等待所有这些副本完成。我在等待中进行了尝试/捕获,但看起来它永远都在等待中被卡住。关于此甚至如何调试这种情况的任何提示?

1 个答案:

答案 0 :(得分:1)

您的代码非常混乱。您正在将异步/等待与Promise.then混合。通常,您会选择一种或另一种方式,但绝不会两者兼有。

通过查看您的代码,我猜测 sendResult返回一个Promise,因为您将其结果分配给了一个名为resultPromise的对象,但是您从未等待它。

>

copyHandler还返回一个Promise,但您没有等待它(您正在链接then调用),但是到return resultPromise时,承诺尚未兑现。

对我来说,问题出在您的cfnHandlerWrapper函数上。我会将您发布的代码更改为:

const cfnHandlerWrapper = async (event, handler) => {
    try {
        await copyHandler(event)
        console.log(`Successfully copied over all artifacts`);
        return await sendSuccess(event);
    } catch (e) {
        console.log('ERROR RUNNING HANDLER:');
        console.log(e);
        return await sendFailure(e.message || 'Something went wrong', event);
    }
};

现在,在您的forEach上您遇到了另一个问题,因为其中的代码包含也会异步运行的promises,因此,在您调用await Promise.all(copyPromises);时,copyPromises尚未被填充拥有您所期望的所有承诺(可能没有任何承诺)。

我会改用for of(下面仅相关代码):

for (const object of listObjectsResult.Contents) {
    console.log(`parsing object ${object}`);
    let keyParts = object.Key.split('/');
    let fileKey = keyParts.pop();
    let destKey = fileKey.includes(agentBootstrapScript) ? `${agentBootstrapScriptsFolder}/${agentBootstrapScript}` : fileKey; // Adhere to bootstrap folder structure
    let copyParams = {
        Bucket: DestBucket,
        CopySource: `${srcLocation}/${fileKey}`,
        Key: destKey
    };
    console.log("pushing copy promise");
    copyPromises.push(s3.copyObject(copyParams).promise())
}
console.log("about to await Promise.all()....");
await Promise.all(copyPromises)

不测试就很难判断这段代码是否有效,但是我不知道为什么它会失败。无论如何,请尝试将您的组件分解成较小的部分,并确切地查看它出故障的地方。不过,我赌我所有的硬币都是在履行承诺的方式上。