Question

我有一个大约1GB的ASCII文件，我的任务是每1000行写一个固定的文本。

例如

预期产量

text line 1
text line 2
text line 3
...
text line 999
text line 1000
// My fixed text
text line 1001
text line 1002

临时1 -JavaScript堆内存不足

尝试一次写入一个字节，并跟踪换行符的数量。

每1000行写我的固定文本

let src = `/path/to/src`;
let dst = `/path/to/dst`;

let readStream = fs.createReadStream(src);
let writeStream = fs.createWriteStream(dst);
let newline = 0;

readStream.on('data', chunk => {
    for(let i in chunk) {
        let c = chunk[i];

        // write one byte at a time 
        writeStream.write(Buffer.from(chunk.buffer, i, 1))

        if(c == 10) {
            newline++;

            // write fixed text for every 1000th line
            if(newline && newline % 1000 == 0) {
                writeStream.write("My Fixed Text");
            }
        }
    }
}).on('end', () => {
    console.log('done');
});

致命错误：接近堆限制分配的无效标记压缩失败-JavaScript堆内存不足

临时2 -输出文件混乱且乱序

与attemp 1相同，但一次从预分配的缓冲区中写入一个字节。

let src = `/path/to/src`;
let dst = `/path/to/dst`;

let readStream = fs.createReadStream(src);
let writeStream = fs.createWriteStream(dst);
let newline = 0;
let buffer = Buffer.alloc(1);

readStream.on('data', chunk => {
    for(let i in chunk) {
        let c = chunk[i];

        buffer[0] = c;

        // write one byte at a time from my pre-allocated buffer
        writeStream.write(buffer)

        if(c == 10) {
            newline++;

            // write fixed text for every 1000th line
            if(newline && newline % 1000 == 0) {
                writeStream.write("My Fixed Text");
            }
        }
    }
}).on('end', () => {
    console.log('done');
});

没有错误，脚本在打印done之后花了一段时间退出

但是，输出文件看起来完全混乱并且混乱了

L,a'-etLL'7P62 ,,NL,n''he'L A,r'0et,L ,ei2'vrUb32oa0on' L,c27CirU 7' 2- tN Ng7Bha'0atLV7al'0e'pUbLe2,3 goLA0n'0,v',L9lc'1C,LI0Et0-m',UT5,nD1e''NU,On91arNE40r20ilHLVEn''''''''mwRL (3(0-r''NUB o00at S7Eo'7'clUR32 20vowL 64v80vaL 0,u00hNNUE4n,1'aa L 2 AC1p,,NU e2'Cit, 22y F8etLL56Ola-1C'NLS3n '3mrUS34nn1-CL,g5tw7ma'L)'4t60h'L

这个任务听起来很简单明了，但是我很难做到。

感谢您的帮助。

如何从巨大的文件中每隔几行插入文本

0 个答案: