如何为GCS WriteStream捕获UnhandledPromiseRejectionWarning

时间:2019-03-15 20:47:36

标签: node.js google-cloud-platform

观察到的应用行为

在node.js中使用UnhandledPromiseRejectionWarning: Error: Upload failed时得到@google-cloud/storage

这些错误在处理数​​千个请求时出现。造成错误的比例很小,但是由于缺乏处理错误的能力以及错误消息中缺少适当的上下文,很难确定WHICH文件是否失败。

我知道一般而言,promise必须具有.catch或被try / catch块包围。但是在这种情况下,我使用的是写流。关于被拒绝的承诺的实际位置以及如何截取它,我有些困惑。堆栈跟踪没有帮助,因为它仅包含库代码:

UnhandledPromiseRejectionWarning: Error: Upload failed
    at Request.requestStream.on.resp (.../node_modules/gcs-resumable-upload/build/src/index.js:163:34)
    at emitTwo (events.js:131:20)
    at Request.emit (events.js:214:7)
    at Request.<anonymous> (.../node_modules/request/request.js:1161:10)
    at emitOne (events.js:121:20)
    at Request.emit (events.js:211:7)
    at IncomingMessage.<anonymous> (.../node_modules/request/request.js:1083:12)
    at Object.onceWrapper (events.js:313:30)
    at emitNone (events.js:111:20)
    at IncomingMessage.emit (events.js:208:7)

我的代码

创建writeStream的代码如下:

const {join} = require('path')
const {Storage} = require('@google-cloud/storage')

module.exports = (config) => {
  const storage = new Storage({
    projectId: config.gcloud.project,
    keyFilename: config.gcloud.auth_file
  })

  return {
    getBucketWS(path, contentType) {
      const {bucket, path_prefix} = config.gcloud

      // add path_prefix if we have one
      if (path_prefix) {
        path = join(path_prefix, path)
      }

      let setup = storage.bucket(bucket).file(path)
      let opts = {}
      if (contentType) {
        opts = {
          contentType,
          metadata: {contentType}
        }
      }
      const stream = setup.createWriteStream(opts)
      stream._bucket = bucket
      stream._path = path
      return stream
    }
  }
}

使用的代码如下:

const gcs = require('./gcs-helper.js')

module.exports = ({writePath, contentType, item}, done) => {
  let ws = gcs.getBucketWS(writePath, contentType)
  ws.on('error', (err) => {
    err.message = `Could not open gs://${ws._bucket}/${ws._path}: ${err.message}`
    done(err)
  })
  ws.on('finish', () => {
    done(null, {
      path: writePath,
      item
    })
  })
  ws.write(item)
  ws.end()
}

鉴于我已经在监听流中的error事件,因此看不到我还能做什么。在我正在使用的@google-cloud/storage级别上没有达成承诺。

进入@ google-cloud / storage库

堆栈跟踪的第一行将我们带到gcs-resumable-upload节点模块中的代码块,如下所示:

requestStream.on('complete', resp => {
    if (resp.statusCode < 200 || resp.statusCode > 299) {
        this.destroy(new Error('Upload failed'));
        return;
    }
    this.emit('metadata', resp.body);
    this.deleteConfig();
    this.uncork();
});

这会将错误传递给流上的destroy方法。 @google-cloud/common project's utility module正在创建流,并且正在使用duplexify node module创建流。 destroy方法在双工化流上定义,可以在README文档中找到。

在阅读the duplexify code时,我看到它先发出this._ondrain的信息,然后才发出错误。也许我可以提供一个回调来避免未解决此错误?

我尝试了ws.write(item, null, cb),但仍然得到了相同的UnhandledPromiseRejectionWarning。我尝试了ws.end(item, null, cb),甚至将.end调用包装在一个try catch中,最终收到此错误,使该过程完全崩溃:

events.js:183
      throw er; // Unhandled 'error' event
      ^

Error: The uploaded data did not match the data from the server. As a precaution, the file has been deleted. To be sure the content is the same, you should try uploading the file again.
    at delete (.../node_modules/@google-cloud/storage/build/src/file.js:1295:35)
    at Util.handleResp (.../node_modules/@google-cloud/common/build/src/util.js:123:9)
    at retryRequest (.../node_modules/@google-cloud/common/build/src/util.js:404:22)
    at onResponse (.../node_modules/retry-request/index.js:200:7)
    at .../node_modules/teeny-request/build/src/index.js:208:17
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:189:7)

我的最终代码如下:

let ws = gcs.getBucketWS(writePath, contentType)
const handleErr = (err) => {
  if (err) err.message = `Could not open gs://${ws._bucket}/${ws._path}: ${err.message}`
  done(err)
}
ws.on('error', handleErr)
// trying to do everything we can to handle these errors
// for some reason we still get UnhandledPromiseRejectionWarning
try {
  ws.write(item, null, err => {
    handleErr(err)
  })
  ws.end()
} catch (e) {
  handleErr(e)
}

结论

对于我来说,@google-cloud/storage库(或duplexify)的用户应该如何执行正确的错误处理仍然是一个谜。来自任何一个项目的图书馆维护者的评论将不胜感激。谢谢!

0 个答案:

没有答案