使用NodeJS将多个文件上载到AWS S3

时间:2017-04-27 16:01:47

标签: node.js amazon-web-services amazon-s3

我尝试使用NodeJS将目录中的所有文件上传到我的S3存储桶。如果我明确地为Key:字段提供文件路径+文字字符串,我一次能够上传一个文件。

以下是我使用的脚本:

var AWS = require('aws-sdk'),
    fs = require('fs');

// For dev purposes only
AWS.config.update({ accessKeyId: '...', secretAccessKey: '...' });

// reg ex to match
var re = /\.txt$/;

// ensure that this file is in the directory of the files you want to run the cronjob on

// ensure that this file is in the directory of the files you want to run the cronjob on
fs.readdir(".", function(err, files) {
    if (err) {
    console.log( "Could not list the directory.", err)
    process.exit( 1 )
    }


    var matches = files.filter( function(text) { return re.test(text) } )
    console.log("These are the files you have", matches)
    var numFiles = matches.length


    if ( numFiles ) {
        // Read in the file, convert it to base64, store to S3

        for( i = 0; i < numFiles; i++ ) {
            var fileName = matches[i]

            fs.readFile(fileName, function (err, data) {
                if (err) { throw err }

                // Buffer Pattern; how to handle buffers; straw, intake/outtake analogy
                var base64data = new Buffer(data, 'binary');


                var s3 = new AWS.S3()
                    s3.putObject({
                       'Bucket': 'noonebetterhaventakenthisbucketnname',
                        'Key': fileName,
                        'Body': base64data,
                        'ACL': 'public-read'
                     }, function (resp) {
                        console.log(arguments);
                        console.log('Successfully uploaded, ', fileName)
                    })
            })

        }

    }

})

它会为尝试上传到S3的每个文件产生此错误:

These are the files you have [ 'test.txt', 'test2.txt' ]
{ '0': null,
  '1': { ETag: '"2cad20c19a8eb9bb11a9f76527aec9bc"' } }
Successfully uploaded,  test2.txt
{ '0': null,
  '1': { ETag: '"2cad20c19a8eb9bb11a9f76527aec9bc"' } }
Successfully uploaded,  test2.txt

修改:使用变量名更新以允许读取密钥而不是matches[i]

为什么只上传test2.txt,如何上传matches变量中的每个文件?

1 个答案:

答案 0 :(得分:7)

参考此Asynchronously reading and caching multiple files in nodejs以获得解决方案。

tl; dr 范围问题 - 需要在变量中包装变量;可以通过为<article> <div class="h"> <div class="pb">user</div> &nbsp; <div class="pb">date <a id="m75813" href="#m75183">#</a></div> </div> <div class="text" onclick="alert(parentNode.parentElement.firstChild.childNodes[1].a.id);"> After clicking on the yellow background, the value from [a id="m75813"] (ie. 'm75813') should apppear. Sentence text.<br> Other sample.<br> Hello world. </div> </article>readFile创建一个函数并在for循环中调用它来完成此操作。

s3.putObject