Question

我为谷歌应用创建了一个后端，如下所示：

backends:
- name: dbops
  options: dynamic

我为它创建了一个管理处理程序：

- url: /backend/.*
  script: backend.app
  login: admin

现在我明白管理员工作应该能够永远运行，并且我正在使用TaskQueue启动这项工作，但由于某种原因我的不是。我的工作就是从更大的表中创建数据存储区中的汇总表。此表包含大约12000条记录，它在开发服务器上处理作业需要几分钟，但它可以正常工作。当我将代码推送到appspot并尝试让它运行相同的工作时，我会看到数据存储区超时。

Traceback (most recent call last):
  File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1536, in __call__
    rv = self.handle_exception(request, response, e)
  File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1530, in __call__
    rv = self.router.dispatch(request, response)
  File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.5.1/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/apps/s~myzencoder/dbops.362541511260492787/backend.py", line 626, in get
    for asset in assets:
  File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/db/__init__.py", line 2314, in next
    return self.__model_class.from_entity(self.__iterator.next())
  File "/base/python27_runtime/python27_lib/versions/1/google/appengine/datastore/datastore_query.py", line 2816, in next
    next_batch = self.__batcher.next()
  File "/base/python27_runtime/python27_lib/versions/1/google/appengine/datastore/datastore_query.py", line 2678, in next
    return self.next_batch(self.AT_LEAST_ONE)
  File "/base/python27_runtime/python27_lib/versions/1/google/appengine/datastore/datastore_query.py", line 2715, in next_batch
    batch = self.__next_batch.get_result()
  File "/base/python27_runtime/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 604, in get_result
    return self.__get_result_hook(self)
  File "/base/python27_runtime/python27_lib/versions/1/google/appengine/datastore/datastore_query.py", line 2452, in __query_result_hook
    self._batch_shared.conn.check_rpc_success(rpc)
  File "/base/python27_runtime/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1224, in check_rpc_success
    raise _ToDatastoreError(err)
Timeout: The datastore operation timed out, or the data was temporarily unavailable.

有人对如何使这项工作有任何建议吗？

Answer 1

虽然后端请求可以运行很长时间，但查询只能运行60秒。您必须使用游标循环查询结果。

Mapreduce可以通过并行执行查询来更快地获得结果。

Answer 2

在生产中，您使用HR数据存储区，并且可能会遇到争用问题。看到这篇文章。 https://developers.google.com/appengine/articles/scaling/contention?hl=nl

看看mapreduce来创建报告。也许这是一个更好的解决方案。

管理员任务上的Google应用引擎数据存储超时

2 个答案: