使用fs.createWriteStream在Nodejs中编写大文件

时间:2018-05-21 20:09:44

标签: javascript node.js

我有功能,在完美世界中应该创建一个巨大的1M行文件。这是:

    const fileWriteStream = fs.createWriteStream(path.resolve(filePath));
    let ableToWrite = true;
    for (let i = 0; i < 1e6; i++) {
        if (ableToWrite) {
            ableToWrite = fileWriteStream.write(`.testClass${itr}-${i%2} { background: red } \n`);
        } else {
            fileWriteStream.once('drain', () => {
                ableToWrite = fileWriteStream.write(`.testClass${itr}-${i%2} { background: red } \n`);
            })
        }
    }

对我来说不幸的是,我很快得到了以下错误:

MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 drain listeners added. Use emitter.setMaxListeners() to increase limit

我真的不想增加听众的数量。 使用流编写这么大的文件的正确方法是什么?

非常感谢!

1 个答案:

答案 0 :(得分:2)

当流消失时异步继续的最简单方法是在循环中使用await并带有一个承诺:

const fileWriteStream = fs.createWriteStream(path.resolve(filePath));
for (let i = 0; i < 1e6; i++) {
    const ableToWrite = fileWriteStream.write(`.testClass${itr}-${i%2} { background: red } \n`);
    if (!ableToWrite) {
        await new Promise(resolve => {
            fileWriteStream.once('drain', resolve);
        });
    }
}

替代方法是使用递归方法而不是循环:

function go(i) {
    if (i >= 1e6) return;
    const ableToWrite = fileWriteStream.write(`.testClass${itr}-${i%2} { background: red } \n`);
    if (ableToWrite)
        go(i+1);
    else
        fileWriteStream.once('drain', () => {
            go(i+1);
        });
}
const fileWriteStream = fs.createWriteStream(path.resolve(filePath));
go(0);