修改

Question

我有一个使用Firestore作为队列的应用程序。数据来自其他来源，存储在/data/{id}的Firestore中，属性unprocessed设置为true。在我的Node.js脚本中，我想要一次查询这些未处理的记录，以便慢慢处理它们。有数十万条记录，因此尝试将它们全部加载到内存中会导致进程崩溃。

代码：

firestore.collection('data').where('unprocessed', '==', true).limit(25).onSnapshot((snapshot) => {
    snapshot.forEach((doc) => {
      processItem(doc);
    });
  });

processItem()函数执行必要的处理，将数据保存回Firestore，并将unprocessed属性设置为false。

我遇到的问题是，每当我尝试运行此代码时，都会出现以下错误：

Error: Error 3: Order must include __name__
    at sendError (C:\[myApp]\node_modules\@google-cloud\firestore\src\watch.js:254:15)
    at DestroyableTransform.stream.on.proto (C:\[myApp]\node_modules\@google-cloud\firestore\src\watch.js:532:13)
    at emitOne (events.js:115:13)
    at DestroyableTransform.emit (events.js:210:7)
    at addChunk (C:\[myApp]\node_modules\readable-stream\lib\_stream_readable.js:284:12)
    at readableAddChunk (C:\[myApp]\node_modules\readable-stream\lib\_stream_readable.js:271:11)
    at DestroyableTransform.Readable.push (C:\[myApp]\node_modules\readable-stream\lib\_stream_readable.js:238:10)
    at DestroyableTransform.Transform.push (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:146:32)
    at afterTransform (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:102:51)
    at TransformState.afterTransform (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:79:12)
    at DestroyableTransform.noop [as _transform] (C:\[myApp]\node_modules\through2\through2.js:26:3)
    at DestroyableTransform.Transform._read (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:182:10)
    at DestroyableTransform.Transform._write (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:170:83)
    at doWrite (C:\[myApp]\node_modules\readable-stream\lib\_stream_writable.js:406:64)
    at writeOrBuffer (C:\[myApp]\node_modules\readable-stream\lib\_stream_writable.js:395:5)
    at DestroyableTransform.Writable.write (C:\[myApp]\node_modules\readable-stream\lib\_stream_writable.js:322:11)

从我的代码中删除.limit()使其运行，但后来又遇到另一个问题，即处理和存储项目的功能没有正确完成，我的内存使用量一直在增长，直到进程崩溃。

我的第一直觉是认为.limit()由于某种原因与.onSnapshot()不兼容，但也许有人可以让我更好地了解这里发生了什么。

修改

我还尝试添加.orderBy('__name__')，如https://firebase.google.com/docs/firestore/manage-data/delete-data中所示（在“删除集合”部分中），但这只会导致另一个错误：

Error: Trying to compare documents on fields that don't exist. Please include the fields you are ordering on in your select() call.
    at C:\[myApp]\node_modules\@google-cloud\firestore\src\reference.js:1679:19
    at Array.sort (native)
    at computeSnapshot (C:\[myApp]\node_modules\@google-cloud\firestore\src\watch.js:438:20)
    at push (C:\[myApp]\node_modules\@google-cloud\firestore\src\watch.js:469:18)
    at DestroyableTransform.stream.on.proto (C:\[myApp]\node_modules\@google-cloud\firestore\src\watch.js:514:15)
    at emitOne (events.js:115:13)
    at DestroyableTransform.emit (events.js:210:7)
    at addChunk (C:\[myApp]\node_modules\readable-stream\lib\_stream_readable.js:284:12)
    at readableAddChunk (C:\[myApp]\node_modules\readable-stream\lib\_stream_readable.js:271:11)
    at DestroyableTransform.Readable.push (C:\[myApp]\node_modules\readable-stream\lib\_stream_readable.js:238:10)
    at DestroyableTransform.Transform.push (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:146:32)
    at afterTransform (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:102:51)
    at TransformState.afterTransform (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:79:12)
    at DestroyableTransform.noop [as _transform] (C:\[myApp]\node_modules\through2\through2.js:26:3)
    at DestroyableTransform.Transform._read (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:182:10)
    at DestroyableTransform.Transform._write (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:170:83)

Answer 1

如果您添加.orderBy(FirebaseFirestore.FieldPath.documentId())，则应解决问题。对此问题的简要解释是here。这是一个gist的例子。

将.limit（）与Firestore查询一起使用

修改

1 个答案: