优化FS writeFile以避免大文件大小导致堆满

时间:2019-07-03 23:25:57

标签: javascript node.js fs

主要我需要打开json文件,将数据推送到该文件,然后再次保存。 重复处理大量数据。

问题是我无法完成全部阅读。

我已经测试了以下代码:

var array = require('./test.json')

for (let i = 0;i<999999999999;i++){
array.push(i)

}

fs.writeFile('./test.json', array, function (err) {
    if (err) {
        return console.log(err);
    }

    console.log('saved');
});

它返回:

<--- Last few GCs --->

[3632:000001B9DE56BD10]     2076 ms: Mark-sweep 577.7 (585.1) -> 577.7 (585.1) MB, 92.5 / 0.0 ms  allocation failure GC in old space requested
[3632:000001B9DE56BD10]     2174 ms: Mark-sweep 577.7 (585.1) -> 577.7 (582.1) MB, 97.6 / 0.0 ms  last resort GC in old space requested
[3632:000001B9DE56BD10]     2273 ms: Mark-sweep 577.7 (582.1) -> 577.7 (582.1) MB, 99.3 / 0.0 ms  last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 00000364863257C1 <JSObject>
    1: /* anonymous */ [C:\test\test.js:~1] [pc=000002501A885DE4](this=00000101E000A879 <Object map = 000001EDFF6023B9>,exports=00000101E000A879 <Object map = 000001EDFF6023B9>,require=00000101E000A831 <JSFunction require (sfi = 000000F3AFACAB59)>,module=00000101E000A7A9 <Module map = ...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node_module_register
 2: v8::internal::FatalProcessOutOfMemory
 3: v8::internal::FatalProcessOutOfMemory
 4: v8::internal::Factory::NewUninitializedFixedArray
 5: v8::internal::WasmDebugInfo::SetupForTesting
 6: v8::internal::interpreter::BytecodeArrayRandomIterator::UpdateOffsetFromIndex
 7: 000002501A7043C1

我的问题是:

如何优化代码以避免这些内存问题?

感谢帮助!

0 个答案:

没有答案