我最近实现了一个可写流,它将来自多个可读流的聚合数据持久保存到mongo数据库中。由于流的高吞吐量,我希望实现一些去抖功能。
我正在努力确定这样做的好方法,麻烦来自需要在管道中施加背压。基本上,如果去抖动超时执行,可写的' next()'只有在超时功能完成后才需要调用回调,但是如果超时尚未执行,则可写入需要' next()'立即保持流动。
我觉得我在这里错过了一些简单的东西,希望这里有人可以让我自己踢。
_write()逻辑解释/ TLDR:
// Queues a mongo bulk upsert operation.
queueMongoUpsertOperation(someData);
// This should only be called NOW if the timeout isn't executing
// as we should wait for it to finish the save before reading
// more data IF it is saving.
// Best/SAFEST way to block this call when the timeout is executing
next()
// Simple debounce.
clearTimeout(timeout);
timeout = setTimeout(function(){
// Execute the actual asyncronous save from the queued upsert operations.
executeMongoBulkUpsert(function(){
// Because the next() above is somehow blocked when the timeout
// is executed, we should now called next() from within
// the debounced function.
next();
});
}, 1000);