Question

在Node.js项目中，我试图从S3获取数据。

当我使用getSignedURL时，一切正常：

aws.getSignedUrl('getObject', params, function(err, url){
    console.log(url); 
});

我的参数是：

var params = {
              Bucket: "test-aws-imagery", 
              Key: "TILES/Level4/A3_B3_C2/A5_B67_C59_Tiles.par"

如果我将URL输出带到控制台并将其粘贴到Web浏览器中，它会下载我需要的文件。

但是，如果我尝试使用getObject，我会遇到各种奇怪的行为。我相信我只是错误地使用它。这就是我尝试过的：

aws.getObject(params, function(err, data){
    console.log(data); 
    console.log(err); 
});

输出：

{ 
  AcceptRanges: 'bytes',
  LastModified: 'Wed, 06 Apr 2016 20:04:02 GMT',
  ContentLength: '1602862',
  ETag: '9826l1e5725fbd52l88ge3f5v0c123a4"',
  ContentType: 'application/octet-stream',
  Metadata: {},
  Body: <Buffer 01 00 00 00  ... > }

  null

所以看起来这是正常的。但是，当我在其中一个console.log上放置断点时，我的IDE（NetBeans）会抛出错误并拒绝显示数据的值。虽然这可能只是IDE，但我决定尝试其他方式来使用getObject。

aws.getObject(params).on('httpData', function(chunk){
    console.log(chunk); 
}).on('httpDone', function(data){
    console.log(data); 
});

这不会输出任何内容。在断点处显示代码永远不会到达console.log中的任何一个。我也尝试过：

aws.getObject(params).on('success', function(data){
    console.log(data); 
});

但是，这也没有输出任何内容，并且放置断点表明永远不会达到console.log。

我做错了什么？

Answer 1

从S3 API执行getObject()时，根据docs文件内容位于Body属性中，您可以从示例输出中看到该属性。您应该拥有类似于以下内容的代码

const aws = require('aws-sdk');
const s3 = new aws.S3(); // Pass in opts to S3 if necessary

var getParams = {
    Bucket: 'abc', // your bucket name,
    Key: 'abc.txt' // path to the object you're looking for
}

s3.getObject(getParams, function(err, data) {
    // Handle any error and exit
    if (err)
        return err;

  // No error happened
  // Convert Body from a Buffer to a String

  let objectData = data.Body.toString('utf-8'); // Use the encoding necessary
});

您可能不需要从data.Body对象创建新缓冲区，但如果需要，可以使用上面的示例来实现。

Answer 2

基于@peteb的答案，但使用Promises和Async/Await：

const AWS = require('aws-sdk');

const s3 = new AWS.S3();

async function getObject (bucket, objectKey) {
  try {
    const params = {
      Bucket: bucket,
      Key: objectKey 
    }

    const data = await s3.getObject(params).promise();

    return data.Body.toString('utf-8');
  } catch (e) {
    throw new Error(`Could not retrieve file from S3: ${e.message}`)
  }
}

// To retrieve you need to use `await getObject()` or `getObject().then()`
getObject('my-bucket', 'path/to/the/object.txt').then(...);

Answer 3

或者，您可以使用minio-js client library get-object.js

var Minio = require('minio')

var s3Client = new Minio({
  endPoint: 's3.amazonaws.com',
  accessKey: 'YOUR-ACCESSKEYID',
  secretKey: 'YOUR-SECRETACCESSKEY'
})

var size = 0
// Get a full object.
s3Client.getObject('my-bucketname', 'my-objectname', function(e, dataStream) {
  if (e) {
    return console.log(e)
  }
  dataStream.on('data', function(chunk) {
    size += chunk.length
  })
  dataStream.on('end', function() {
    console.log("End. Total size = " + size)
  })
  dataStream.on('error', function(e) {
    console.log(e)
  })
})

免责声明：我为Minio工作，使用golang编写的开源S3兼容对象存储，Java，Python，Js，{{3}提供客户端库}}。

Answer 4

乍一看，看起来你做错了什么但是你没有显示你的所有代码。当我第一次检查S3和Node时，以下内容对我有用：

var AWS = require('aws-sdk');

if (typeof process.env.API_KEY == 'undefined') {
    var config = require('./config.json');
    for (var key in config) {
        if (config.hasOwnProperty(key)) process.env[key] = config[key];
    }
}

var s3 = new AWS.S3({accessKeyId: process.env.AWS_ID, secretAccessKey:process.env.AWS_KEY});
var objectPath = process.env.AWS_S3_FOLDER +'/test.xml';
s3.putObject({
    Bucket: process.env.AWS_S3_BUCKET, 
    Key: objectPath,
    Body: "<rss><data>hello Fred</data></rss>",
    ACL:'public-read'
}, function(err, data){
    if (err) console.log(err, err.stack); // an error occurred
    else {
        console.log(data);           // successful response
        s3.getObject({
            Bucket: process.env.AWS_S3_BUCKET, 
            Key: objectPath
        }, function(err, data){
            console.log(data.Body.toString());
        });
    }
});

Answer 5

对于正在寻找上述版本NEST JS TYPESCRIPT的用户：

    /**
     * to fetch a signed URL of a file
     * @param key key of the file to be fetched
     * @param bucket name of the bucket containing the file
     */
    public getFileUrl(key: string, bucket?: string): Promise<string> {
        var scopeBucket: string = bucket ? bucket : this.defaultBucket;
        var params: any = {
            Bucket: scopeBucket,
            Key: key,
            Expires: signatureTimeout  // const value: 30
        };
        return this.account.getSignedUrlPromise(getSignedUrlObject, params);
    }

    /**
     * to get the downloadable file buffer of the file
     * @param key key of the file to be fetched
     * @param bucket name of the bucket containing the file
     */
    public async getFileBuffer(key: string, bucket?: string): Promise<Buffer> {
        var scopeBucket: string = bucket ? bucket : this.defaultBucket;
        var params: GetObjectRequest = {
            Bucket: scopeBucket,
            Key: key
        };
        var fileObject: GetObjectOutput = await this.account.getObject(params).promise();
        return Buffer.from(fileObject.Body.toString());
    }

    /**
     * to upload a file stream onto AWS S3
     * @param stream file buffer to be uploaded
     * @param key key of the file to be uploaded
     * @param bucket name of the bucket 
     */
    public async saveFile(file: Buffer, key: string, bucket?: string): Promise<any> {
        var scopeBucket: string = bucket ? bucket : this.defaultBucket;
        var params: any = {
            Body: file,
            Bucket: scopeBucket,
            Key: key,
            ACL: 'private'
        };
        var uploaded: any = await this.account.upload(params).promise();
        if (uploaded && uploaded.Location && uploaded.Bucket === scopeBucket && uploaded.Key === key)
            return uploaded;
        else {
            throw new HttpException("Error occurred while uploading a file stream", HttpStatus.BAD_REQUEST);
        }
    }

Answer 6

这是 async / await 版本

var getObjectAsync = async function(bucket,key) {
  try {
    const data = await s3
      .getObject({ Bucket: bucket, Key: key })
      .promise();
      var contents = data.Body.toString('utf-8');
      return contents;
  } catch (err) {
    console.log(err);
  }
}
var getObject = async function(bucket,key) {
    const contents = await getObjectAsync(bucket,key);
    console.log(contents.length);
    return contents;
}
getObject(bucket,key);

Answer 7

与上面@ArianAcosta 的答案极其相似。除了我使用 import（适用于 Node 12.x 及更高版本），添加 AWS 配置和嗅探图像有效负载并将 base64 处理应用于 return。

// using v2.x of aws-sdk
import aws from 'aws-sdk'

aws.config.update({
  accessKeyId: process.env.YOUR_AWS_ACCESS_KEY_ID,
  secretAccessKey: process.env.YOUR_AWS_SECRET_ACCESS_KEY,
  region: "us-east-1" // or whatever
})

const s3 = new aws.S3();

/**
 * getS3Object()
 * 
 * @param { string } bucket - the name of your bucket
 * @param { string } objectKey - object you are trying to retrieve
 * @returns { string } - data, formatted
 */
export async function getS3Object (bucket, objectKey) {
  try {
    const params = {
      Bucket: bucket,
      Key: objectKey 
    }

    const data = await s3.getObject(params).promise();

    // Check for image payload and formats appropriately
    if( data.ContentType === 'image/jpeg' ) {
      return data.Body.toString('base64');
    } else {
      return data.Body.toString('utf-8');
    }

  } catch (e) {
    throw new Error(`Could not retrieve file from S3: ${e.message}`)
  }
}

Answer 8

也许此示例可以帮助需要返回文件（如果存在）或JSON响应的人，以防在存储桶名称或文件为：时需要传递JSON的情况

  // Assuming you already connected to AWS SDK and S3 

  var bucketParams = {
      Bucket: MyBucketName, //if variable or dynamically created otherwise string
      Key: myKey //if variable or dynamically created otherwise string
  };

  s3.getObject(bucketParams, function(err, data) {
        if (err) {
        //this will check connection and file / bucket existance
          console.log(err);
          return res.json({
            success: false,
            error: err
          });
        } else {
          res.attachment(key);
          s3.getObject(bucketParams)
            .createReadStream()
            .pipe(res);
        }
      });

Answer 9

假设我的 S3 对象如下所示

现在我想从存储桶访问文件“ElasticsearchConfigNew.json” 在访问 S3 之前，您必须获得许可。定义策略并将其附加到角色。

Nodejs 代码

let AWS = require('aws-sdk');
let s3 = new AWS.S3();
 exports.handler = async (event) => {
  await getFromS3().then((message) => {
console.log("After success", message);
}).catch((message) => {
console.log("After Failure",message);
});
};



async function getFromS3 (){
await new Promise((resolve, reject) => {

        console.log(" Accessing to S3 ");
        
        let lambda_params = {
            Bucket: 'px-library', //S3 Bucket Name
            Key:   "ElasticsearchConfigNew.json" //File present inside S3 Bucket
            };

    // Call S3 to obtain a list of the objects in the bucket
    s3.getObject(lambda_params, function(err, data) {
      if (err) {
        console.log("Error", err);
        reject(err);
      } else {
        console.log("Success  : ",data.Body.toString());
        resolve(data.Body.toString());
      }
    });
  });
  }

如何从Node.js中的S3 getObject获取响应？

9 个答案: