延期任务请求截止日期已超出,但工作从未开始

时间:2015-06-18 14:43:46

标签: python google-app-engine task-queue

我有一个任务队列来处理需要实时处理的延期任务(我听说这不是一个好主意但无法找到任何支持文档。)。这个队列被大量涌入的任务(2500 / min)击中,然后经常在一两分钟内没有收到任何结果。

queue.yaml中

- name: event-message
  rate: 200/s
  bucket_size: 200
  retry_parameters:
      task_retry_limit: 2
      min_backoff_seconds: 1

我注意到的是,我经常因为超出请求截止日期而终止了流程。 (错误代码123)"某些甚至无法启动的任务的消息。任务在队列中停留10分钟而不运行,然后超时。我认为它可能与我的队列配置有关,所以我在这里包含了它。

2015-06-18 10:22:45.842 /_ah/queue/deferred 500 641080ms 0kb AppEngine-Google; (+http://code.google.com/appengine) module=default version=1-0-21
0.1.0.2 - - [18/Jun/2015:07:22:45 -0700] "POST /_ah/queue/deferred HTTP/1.1" 500 0 "http://*******/_ah/queue/deferred" "AppEngine-Google; (+http://code.google.com/appengine)" "********" ms=641081 cpu_ms=0 queue_name=event-message task_name=0477163503045474472 exit_code=123 app_engine_release=1.9.22 instance=00c61b117c8af1f4df1247032afd14111a19f302
E 2015-06-18 10:22:45.842
Process terminated because the request deadline was exceeded. (Error code 123)

任何人都能解释一下吗?任何想法我怎么能弄清楚为什么这些任务永远不会运行?

所以我更多地查看日志,并发现对我的理论的额外支持,即任务甚至没有开始。这里"此请求导致为您的应用程序启动一个新进程"在任务完成10分钟截止日期后出现。

2015-06-24 17:16:59.554 /_ah/queue/deferred 500 641631ms 0kb AppEngine-Google; (+http://code.google.com/appengine) module=default version=1-0-27
0.1.0.2 - - [24/Jun/2015:14:16:59 -0700] "POST /_ah/queue/deferred HTTP/1.1" 500 0 "https://integrate.versature.com/api/netsapiens_events/96615760-cac7-40d6-b80d-13a1ffade2c8/" "AppEngine-Google; (+http://code.google.com/appengine)" "integrate.versature.com" ms=641632 cpu_ms=0 queue_name=event-message task_name=59067488044178506841 pending_ms=1061 exit_code=123 app_engine_release=1.9.22 instance=00c61b117c8f2db4ade6be8d30165cb84f0a17ab
E 2015-06-24 17:16:59.554
Process terminated because the request deadline was exceeded. (Error code 123)

这是另一个:

I 2015-06-23 13:48:22.970
X-Appengine-Taskretrycount:0, X-Appengine-Tasketa:1435081701.5576639, X-Appengine-Default-Namespace:versature.com, X-Appengine-Taskname:89115825957976957021, X-Appengine-Taskexecutioncount:0, X-Appengine-Queuename:event-message, X-Appengine-Current-Namespace:, X-Appengine-Country:ZZ

E 2015-06-23 13:58:21.669
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 267, in Handle
result = handler(dict(self._environ), self._StartResponse)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1505, in __call__
rv = self.router.dispatch(request, response)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1253, in default_dispatcher
return route.handler_adapter(request, response)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1077, in __call__
return handler.dispatch()
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 545, in dispatch
return method(*args, **kwargs)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 310, in post
self.run_from_request()
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 305, in run_from_request
run(self.request.body)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 147, in run
return func(*args, **kwds)
File "/base/data/home/apps/s~versature-integrate-2/1-0-26.385226646541088482/app/api/resources/constructs/netsapiens_event/process_netsapiens_event.py", line 66, in process_data_for_endpoint
logging.info("Process data for specific endpoint. Endpoint Key: %s", endpoint_key_url_safe)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/logging/__init__.py", line 1611, in info
root.info(msg, *args, **kwargs)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/logging/__init__.py", line 1149, in info
self._log(INFO, msg, args, **kwargs)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/logging/__init__.py", line 1268, in _log
self.handle(record)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/logging/__init__.py", line 1278, in handle
self.callHandlers(record)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/logging/__init__.py", line 1318, in callHandlers
hdlr.handle(record)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/logging/__init__.py", line 749, in handle
self.emit(record)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/app_logging.py", line 79, in emit
self._AppLogsLocation())
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/logservice/logservice.py", line 457, in write_record
logs_buffer().write_record(level, created, message, source_location)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/logservice/logservice.py", line 270, in write_record
message = cleanup_message(message)
DeadlineExceededError

I 2015-06-23 13:58:21.684
This request caused a new process to be started for your application, and thus caused your application code to be loaded for the first time. This request may thus take longer and use more CPU than a typical request for your application.

0 个答案:

没有答案