我正在生成一个大对象并尝试将它们安全地保存到.json
文件中。但是,即使使用流式传输,看起来该对象正在缓存堆并且永远不会被释放。
这是我称之为的功能:
saveObjectAsJSON (o: any, filename: string) {
const content = JSON.stringify(o);
const wstream = fs.createWriteStream(filename);
wstream.write(content);
wstream.end();
}
它适用于较小的物体,但我现在生成相当大的物体(在外层约有100万个物体)。这是否达到了Node的容量和功能?我应该使用Mongo或其他东西来存储/检索它们吗?
错误追踪:
<--- Last few GCs --->
[6583:0x102801e00] 474566 ms: Mark-sweep 2948.9 (4039.7) -> 2948.8 (4024.7) MB, 6035.6 / 0.0 ms allocation failure GC in old space requested
[6583:0x102801e00] 480764 ms: Mark-sweep 2948.8 (4024.7) -> 2948.8 (3993.7) MB, 6198.0 / 0.0 ms last resort GC in old space requested
[6583:0x102801e00] 487025 ms: Mark-sweep 2948.8 (3993.7) -> 2948.8 (3963.7) MB, 6260.8 / 0.0 ms last resort GC in old space requested
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 0x934a4ba5529 <JSObject>
1: fromString(aka fromString) [buffer.js:336] [bytecode=0x9340cbd6b51 offset=161](this=0x934c6e822d1 <undefined>,string=0x9350da7d189 <Very long string[936824012]>,encoding=0x934a4bb6419 <String[4]: utf8>)
2: from [buffer.js:189] [bytecode=0x9340cbd6611 offset=14](this=0x934616bd3b1 <JSFunction Buffer (sfi = 0x934e8b83821)>,value=0x9350da7d189 <Very long string[936824012]>,encodingOrOffse...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node::Abort() [/usr/local/bin/node]
2: node::FatalTryCatch::~FatalTryCatch() [/usr/local/bin/node]
3: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/usr/local/bin/node]
4: v8::internal::Factory::NewRawOneByteString(int, v8::internal::PretenureFlag) [/usr/local/bin/node]
5: v8::internal::String::SlowFlatten(v8::internal::Handle<v8::internal::ConsString>, v8::internal::PretenureFlag) [/usr/local/bin/node]
6: v8::String::WriteUtf8(char*, int, int*, int) const [/usr/local/bin/node]
7: node::StringBytes::Write(v8::Isolate*, char*, unsigned long, v8::Local<v8::Value>, node::encoding, int*) [/usr/local/bin/node]
8: node::Buffer::New(v8::Isolate*, v8::Local<v8::String>, node::encoding) [/usr/local/bin/node]
9: node::Buffer::(anonymous namespace)::CreateFromString(v8::FunctionCallbackInfo<v8::Value> const&) [/usr/local/bin/node]
10: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) [/usr/local/bin/node]
11: v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) [/usr/local/bin/node]
12: v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) [/usr/local/bin/node]
13: 0x39ffa47042fd
14: 0x39ffa47bd196