将xlsx模块与大文件一起使用时出现内存不足异常

时间:2018-08-28 07:35:42

标签: node.js memory-leaks xlsx

我正在尝试获取Excel工作表中存在的行数和列数

const xlsx = require('xlsx')
let dothis= (path) =>{
    let workbook = xlsx.readFile(path)
    let sheetsList = workbook.SheetNames

    for (let i in sheetsList) {
        var ref = workbook.Sheets[sheetsList[i]]["!ref"];
        var range = xlsx.utils.decode_range(ref);
        console.log(range);
    }
}
dothis('../storage/1million records- Copy.xlsx')

如果文件包含较少的记录,则表明工作正常。但是,如果其中包含100万条记录,则会出现以下错误:

<--- Last few GCs --->

[10020:00000000004E7540]    88573 ms: Mark-sweep 1110.2 (1341.3) ->
1110.2 (1341.3) MB, 4542.4 / 0.0 ms  allocation failure GC in old space requested [10020:00000000004E7540]    92546 ms: Mark-sweep
1110.2 (1341.3) -> 1110.2 (1326.3) MB, 3973.0 / 0.0 ms  last resort GC in old space requested [10020:00000000004E7540]    97207 ms: Mark-sweep 1110.2 (1326.3) -> 1110.2 (1317.8) MB, 4660.5 / 0.0 ms  last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 00000350676A57C1 <JSObject>
    1: parse_ws_xml_data [e:\eg\node_modules\xlsx\xlsx.js:~12315] [pc=00000280234400A9](this=000003CD0C00C211 <JSGlobal Object>,sdata=000002BA7A4A2089 <Very long string[287008182]>,s=000002BA7A4A21B1 <Object map = 00000294315E5519>,opts=000003279F33D969 <Object map = 00000294315E5CA9>,guess=000002BA7A4A2189 <Object map = 00000294315E9401>,themes=000003279F33DA41 <Object map = 00000294315...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory  1: node_module_register  2: v8::internal::FatalProcessOutOfMemory  3: v8::internal::FatalProcessOutOfMemory  4: v8::internal::Factory::NewFixedArray  5: v8::internal::Dictionary<v8::internal::SeededNumberDictionary,v8::internal::SeededNumberDictionaryShape>::AtPut 6: v8::internal::HashTable<v8::internal::SeededNumberDictionary,v8::internal::SeededNumberDictionaryShape>::IsKey 7: v8::internal::HashTable<v8::internal::SeededNumberDictionary,v8::internal::SeededNumberDictionaryShape>::IsKey 8: v8::internal::LookupIterator::ApplyTransitionToDataProperty  9: v8::internal::JSReceiver::class_name 10: v8::internal::SourcePositionTableIterator::Advance 11: v8::internal::SourcePositionTableIterator::Advance 12: 00000280232843C1

1 个答案:

答案 0 :(得分:1)

您可能希望使用--max-old-space-size标志运行脚本并分配更多内存。

node --max-old-space-size=8000 hello.js

将为当前正在运行的实例分配8GB内存。