Google App脚本-使用getAll方法时,缓存服务会丢失键

时间:2018-08-30 08:53:31

标签: google-apps-script google-data-studio

在我的脚本中,我使用缓存服务来存储分成100KB块的大字符串。我最近发现了getAll方法,该方法可以大大加快速度。不幸的是,有时它是不可靠的-尝试一次加载许多密钥时,它往往会丢失密钥(大多数情况下,当密钥数目超过200时)。有趣的是,如果我收集丢失的密钥并仅使用丢失的密钥再次运行getAll,则会成功加载所有密钥。

您是否遇到过类似的问题?通过几次getAll运行收集数据可以解决我的问题,但是我很好奇是否存在任何未指定的限制。

如果有人感兴趣,这是我使用getAll加载数据的方法:

PaginatedCache.prototype._getChunksAsync = function(expectedChunkKeys) {
  var expectedChunkCount = expectedChunkKeys.length;
  var fetchedChunks = {};
  var fetchedChunkKeys = [];

  // This is done in loop, because it sometimes happens that Google cache's getAll
  // method randomly missing some keys, most often when the number of keys is greater than 200.
  // Fetching the missing keys again using getAll solves the problem.
  while (fetchedChunkKeys.length < expectedChunkCount) {
    console.log(expectedChunkCount - fetchedChunkKeys.length, 'keys need to be loaded');
    var missingKeys = this._getMissingKeys(fetchedChunkKeys, expectedChunkKeys);
    var fetchedMissingChunks = this.service.getAll(missingKeys);
    var fetchedMissingChunkKeys = Object.keys(fetchedMissingChunks);

    if (fetchedMissingChunkKeys.length === 0) {
      console.log('0 chunks loaded');
    }

    for (var missingChunkKey in fetchedMissingChunks) {
      fetchedChunks[missingChunkKey] = fetchedMissingChunks[missingChunkKey];
    }

    console.log('Loaded', fetchedMissingChunkKeys.length, 'keys');
    fetchedChunkKeys = fetchedChunkKeys.concat(fetchedMissingChunkKeys);
  }

  return fetchedChunks;
};

还请查看我的脚本执行日志: execution log

编辑:我添加了一个带有复制的简单脚本:https://script.google.com/d/1-LkE70P0JJxoTqKGnoCv0mNM6wWHFAWRbFHH1SbWbnQusHuUe53cyRgn/edit?usp=sharing

示例日志:

[18-09-01 15:17:06:396 CEST] Value length: 17888891
[18-09-01 15:17:06:409 CEST] Storing chunks test_data, Chunks: 175
[18-09-01 15:17:06:914 CEST] Chunk count in cache:175
[18-09-01 15:17:06:919 CEST] 175 keys need to be loaded
[18-09-01 15:17:07:278 CEST] Loaded 171 keys
[18-09-01 15:17:07:279 CEST] 4 keys need to be loaded
[18-09-01 15:17:07:282 CEST] Missing keys: ["test_data_PC_UNIQUE_KEY2_163","test_data_PC_UNIQUE_KEY2_167","test_data_PC_UNIQUE_KEY2_171","test_data_PC_UNIQUE_KEY2_174"]
[18-09-01 15:17:07:313 CEST] Loaded 4 keys
[18-09-01 15:17:08:510 CEST] Loaded 175 chunks from cache asynchronously

0 个答案:

没有答案