Lodash记录“ Java堆内存不足”

时间：2018-09-28 20:59:40

标签： javascript algorithm recursion lodash memoization

我正在尝试通过将备忘录应用于递归解决方案来使用Javascript解决LeetCode的longest palindromic subsequence问题。这是递归解决方案longestPalindromicSubsequence.js：

function longestPalindromicSubsequence(string, start = 0, end = string.length) {
  if (end < start) { return 0; }
  if (start === end) { return 1; }
  if (string[start] === string[end]) {
    return 2 + longestPalindromicSubsequence(string, start + 1, end - 1);
  }
  return Math.max(
    longestPalindromicSubsequence(string, start + 1, end),
    longestPalindromicSubsequence(string, start, end - 1),
  );
}

module.exports = longestPalindromicSubsequence;

这里有一些Jest测试用例，longestPalindromicSubsequence.test.js：

const longestPalindromicSubsequence = require('./longestPalindromicSubsequence');

describe('longest palindromic subsequence', () => {
  test('works for aab', () => {
    expect(longestPalindromicSubsequence('aab')).toBe(2);
  });

  test('works for long string', () => {
    expect(longestPalindromicSubsequence(`${'a'.repeat(50)}bcdef`)).toBe(50);
  });
});

这有效，但是由于递归调用数量呈指数增长，所以速度很慢。例如，对于长度为〜50的字符串，它需要9秒：

$ jest longestPalindromicSubsequence.test.js
 PASS  ./longestPalindromicSubsequence.test.js (9.6s)
  longest palindromic subsequence
    ✓ works for aab (3ms)
    ✓ works for long string (9315ms)

Test Suites: 1 passed, 1 total
Tests:       2 passed, 2 total
Snapshots:   0 total
Time:        10.039s
Ran all test suites matching /longestPalindromicSubsequence.test.js/i.

为了提高性能，我尝试在更新的模块longestPalindromicSubsequence2.js中使用_.memoize：

const _ = require('lodash');

const longestPalindromicSubsequence = _.memoize(
  (string, start = 0, end = string.length) => {
    if (end < start) { return 0; }
    if (start === end) { return 1; }
    if (string[start] === string[end]) {
      return 2 + longestPalindromicSubsequence(string, start + 1, end - 1);
    }
    return Math.max(
      longestPalindromicSubsequence(string, start + 1, end),
      longestPalindromicSubsequence(string, start, end - 1),
    );
  },
  (string, start, end) => [string, start, end], // resolver function
);

module.exports = longestPalindromicSubsequence;

但是，当我尝试使用此模块运行测试时，出现“ Javascript堆内存不足”错误：

$ jest longestPalindromicSubsequence.test.js

 RUNS  ./longestPalindromicSubsequence.test.js

<--- Last few GCs --->
at[89308:0x104801e00]    15800 ms: Mark-sweep 1379.2 (1401.3) -> 1379.2 (1401.3) MB, 1720.4 / 0.0 ms  (+ 0.0 ms in 5 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 1735 ms) (average mu = 0.128, current mu = 0.057) allocat[89308:0x104801e00]    17606 ms: Mark-sweep 1390.0 (1412.3) -> 1390.0 (1412.3) MB, 1711.7 / 0.0 ms  (+ 0.0 ms in 4 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 1764 ms) (average mu = 0.091, current mu = 0.052) allocat

<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x20b000bdc01d]
Security context: 0x1c189571e549 <JSObject>
    1: /* anonymous */ [0x1c18f7682201] [/Users/kurtpeek/GoogleDrive/LeetCode/longestPalindromicSubsequence2.js:~14] [pc=0x20b0015cd091](this=0x1c18d38893a1 <JSGlobal Object>,string=0x1c18f7682271 <String[55]: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabcdef>,start=45,end=45)
    2: memoized [0x1c18f7682309] [/Users/kurtpeek/GoogleDrive/LeetCode/node_...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x100037733 node::Abort() [/usr/local/bin/node]
 2: 0x1000378d6 node::FatalTryCatch::~FatalTryCatch() [/usr/local/bin/node]
 3: 0x10018e57b v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 4: 0x10018e51c v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 5: 0x1004682ee v8::internal::Heap::UpdateSurvivalStatistics(int) [/usr/local/bin/node]
 6: 0x100469ed7 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/usr/local/bin/node]
 7: 0x1004675cb v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/local/bin/node]
 8: 0x1004663e6 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node]
 9: 0x10046eafc v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/bin/node]
10: 0x10046eb48 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/bin/node]
11: 0x10044eb7a v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/usr/local/bin/node]
12: 0x100634916 v8::internal::Runtime_AllocateInTargetSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/bin/node]
13: 0x20b000bdc01d 
Abort trap: 6

据Node.js heap out of memory所了解，Node的标准内存使用为1.7GB，我认为这已经足够了。有任何想法为何无法记住该版本，以及如何修复该版本？

2 个答案:

答案 0 :(得分：2)

我设法通过将解析器功能从(string, start, end) => [string, start, end]更改为(string, start, end) => string + start + end来解决了这个问题：

const _ = require('lodash');

const longestPalindromicSubsequence = _.memoize(
  (string, start = 0, end = string.length) => {
    if (end < start) { return 0; }
    if (start === end) { return 1; }
    if (string[start] === string[end]) {
      return 2 + longestPalindromicSubsequence(string, start + 1, end - 1);
    }
    return Math.max(
      longestPalindromicSubsequence(string, start + 1, end),
      longestPalindromicSubsequence(string, start, end - 1),
    );
  },
  (string, start, end) => string + start + end, // resolver function
);

module.exports = longestPalindromicSubsequence;

现在“长字符串”测试只需3毫秒：

$ jest longestPalindromicSubsequence.test.js
 PASS  ./longestPalindromicSubsequence.test.js
  longest palindromic subsequence
    ✓ works for aab (3ms)
    ✓ works for long string (3ms)

Test Suites: 1 passed, 1 total
Tests:       2 passed, 2 total
Snapshots:   0 total
Time:        1.004s, estimated 10s
Ran all test suites matching /longestPalindromicSubsequence.test.js/i.

使用字符串作为缓存中的键似乎比使用数组更节省内存-也许是因为字符串在Javascript中是不可变的？欢迎对此改进做出任何解释。

答案 1 :(得分：2)

我知道您发布了最佳答案，但想补充说明。根本问题是使用数组是造成瓶颈的原因。在幕后，lodash拥有自己的MapCache，它们定义了似乎假设将传递字符串的字符串。

无论如何查看documentation和注释，它们都会将Cache对象公开给您以供您覆盖，前提是该对象与他们的Map具有相同的界面。

  

创建一个记忆func结果的函数。如果解析器是   提供，它将根据以下信息确定用于存储结果的缓存键   提供给记忆功能的参数。默认情况下，第一个   提供给备注功能的参数用作地图缓存   键。 func通过备忘录的this绑定被调用   功能。

     

注意：缓存在备忘中作为缓存属性公开   功能。 可以通过替换来对其创建进行自定义    _。memoize.Cache构造器，该构造器的实例实现清除，删除，获取，具有和设置的Map方法接口。

我进去并测试了您的代码，因为如果要引用键作为对象/非字符串，则应使用的实际Map是WeakMap。这是我测试过的

const _ = require('lodash');

// override Cache and use WeakMap
_.memoize.Cache = WeakMap;

const longestPalindromicSubsequence = _.memoize(
  (string, start = 0, end = string.length) => {
    if (end < start) { return 0; }
    if (start === end) { return 1; }
    if (string[start] === string[end]) {
      return 2 + longestPalindromicSubsequence(string, start + 1, end - 1);
    }
    return Math.max(
      longestPalindromicSubsequence(string, start + 1, end),
      longestPalindromicSubsequence(string, start, end - 1),
    );
  },
  (string, start, end) => [string, start, end], // resolver function
);

module.exports = longestPalindromicSubsequence;

尽管它仍然需要很长时间，但最终最终会通过而不会遇到JavaScript堆内存不足的问题。

如您所见，最好的解决方案是尽管简单地对密钥进行字符串化：）（尽管考虑到@juvian关于使用JSON.stringify的注释，如果最终字符串与字符串末尾相同时，可以使用@ {{1}}）发生碰撞）

相关问题
最新问题