使用文件元数据将节点请求流式传输到云

时间:2018-07-08 19:42:54

标签: node.js express stream koa2 node-streams

我正在使用koa来构建Web应用程序,并且我希望允许用户将文件上传到该应用程序。这些文件需要流式传输到云中,但是我要避免在本地保存文件。

问题是在将上传流传输到可写流之前,我需要一些文件元数据。我想使用mime类型,并可以选择附加其他数据,例如原始文件名等。

我尝试发送带有请求的“ content-type”标头设置为文件类型的二进制数据,但是我希望请求的内容类型为application/octet-stream,所以我可以在后端知道如何处理请求。

我在某处读到,更好的选择是使用multipart/form-data,但是我不确定如何构造请求以及如何解析元数据以便在将管道传递给写入流之前通知云

这是我当前正在使用的代码。基本上,它只是按原样传送请求,而我使用请求标头来了解文件的类型:

module.exports = async ctx => {
  // Generate a random id that will be part of the filename.
  const id = pushid();
  // Get the content type from the header.
  const contentType = ctx.header['content-type'];
  // Get the extension for the file from the content type
  const ext = contentType.split('/').pop();

  // This is the configuration for the upload stream to the cloud.
  const uploadConfig = {
    // I must specify a content type, or know the file extension.
    contentType

    // there is some other stuff here but its not relevant.
  };

  // Create a upload stream for the cloud storage.
  const uploadStream = bucket
    .file(`assets/${id}/original.${ext}`)
    .createWriteStream(uploadConfig);

  // Here is what took me hours to get to work... dev life is hard
  ctx.req.pipe(uploadStream);

  // return a promise so Koa doesn't shut down the request before its finished uploading.
  return new Promise((resolve, reject) =>
    uploadStream.on('finish', resolve).on('error', reject)
  );
};

请假设我对上传协议和流管理不了解很多。

1 个答案:

答案 0 :(得分:0)

好吧,经过大量搜索,我发现有一个解析器可以处理名为busboy的流。它很容易使用,但是在进入代码之前,我强烈建议每个处理read this articlemultipart/form-data请求的人。

这是我解决的方法:

const Busboy = require('busboy');

module.exports = async ctx => {
  // Init busboy with the headers of the "raw" request.
    const busboy = new Busboy({ headers: ctx.req.headers });

    busboy.on('file', (fieldname, stream, filename, encoding, contentType) => {
        const id = pushid();
        const ext = path.extname(filename);
        const uploadStream = bucket
            .file(`assets/${id}/original${ext}`)
            .createWriteStream({
                contentType,
                resumable: false,
                metadata: {
                    cacheControl: 'public, max-age=3600'
                }
            });

        stream.pipe(uploadStream);
    });

  // Pipe the request to busboy.
    ctx.req.pipe(busboy);

  // return a promise that resolves to whatever you want
    ctx.body = await new Promise(resolve => {
        busboy.on('finish', () => {
            resolve('done');
        });
    });
};