我有一个API，它通过result = task.delay()开始一个芹菜任务，然后通过result.get(timeout=5)等待一个结果。我目前正在编写性能测试，该测试实际上经常执行此任务。它在我的本地计算机上运行良好，但是在我们的开发VM中执行时却显示出奇怪的行为。执行约90-92次后，result.get(timeout=5)超时，即使任务在几毫秒内成功完成。
结果似乎在结果后端丢失了。我正在使用RabbitMQ作为两个方向的消息代理：
celery_broker_url = pyamqp://guest@localhost//
celery_result_backend = rpc://
有人可以给我提示如何进一步调查此问题吗？是否可以检查结果是否传递到结果后端？ RabbitMQ日志不显示任何条目：
-- Logs begin at Wed 2019-01-30 16:49:24 UTC, end at Thu 2019-01-31 14:01:46 UTC. --
-- No entries --
以下是完整的堆栈跟踪信息，以帮助您：
[2019-01-31 13:56:42,313] ERROR in app: Exception on /user/lmhsqs/register [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/celery/backends/async.py", line 255, in _wait_for_pending
on_interval=on_interval):
File "/usr/local/lib/python3.6/dist-packages/celery/backends/async.py", line 54, in drain_events_until
raise socket.timeout()
socket.timeout
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.6/dist-packages/flask/_compat.py", line 33, in reraise
raise value
File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1598, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/local/lib/python3.6/dist-packages/connexion/decorators/decorator.py", line 66, in wrapper
response = function(request)
File "/usr/local/lib/python3.6/dist-packages/connexion/decorators/validation.py", line 122, in wrapper
response = function(request)
File "/usr/local/lib/python3.6/dist-packages/connexion/decorators/validation.py", line 293, in wrapper
return function(request)
File "/usr/local/lib/python3.6/dist-packages/connexion/decorators/decorator.py", line 42, in wrapper
response = function(request)
File "/usr/local/lib/python3.6/dist-packages/connexion/decorators/parameter.py", line 219, in wrapper
return function(**kwargs)
File "/mynedata/lib/api/apicalls.py", line 73, in register_user
res_to_return = result.get(timeout=5)
File "/usr/local/lib/python3.6/dist-packages/celery/result.py", line 224, in get
on_message=on_message,
File "/usr/local/lib/python3.6/dist-packages/celery/backends/async.py", line 188, in wait_for_pending
for _ in self._wait_for_pending(result, **kwargs):
File "/usr/local/lib/python3.6/dist-packages/celery/backends/async.py", line 259, in _wait_for_pending
raise TimeoutError('The operation timed out.')
celery.exceptions.TimeoutError: The operation timed out.
127.0.0.1 - - [2019-01-31 13:56:42] "POST /user/lmhsqs/register HTTP/1.1" 500 388 5.050726
[2019-01-31 13:56:47,374] ERROR in app: Exception on /user/lmhsqs/login [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/celery/backends/async.py", line 255, in _wait_for_pending
on_interval=on_interval):
File "/usr/local/lib/python3.6/dist-packages/celery/backends/async.py", line 54, in drain_events_until
raise socket.timeout()
socket.timeout
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.6/dist-packages/flask/_compat.py", line 33, in reraise
raise value
File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1598, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/local/lib/python3.6/dist-packages/connexion/decorators/decorator.py", line 66, in wrapper
response = function(request)
File "/usr/local/lib/python3.6/dist-packages/connexion/decorators/validation.py", line 122, in wrapper
response = function(request)
File "/usr/local/lib/python3.6/dist-packages/connexion/decorators/validation.py", line 293, in wrapper
return function(request)
File "/usr/local/lib/python3.6/dist-packages/connexion/decorators/decorator.py", line 42, in wrapper
response = function(request)
File "/usr/local/lib/python3.6/dist-packages/connexion/decorators/parameter.py", line 219, in wrapper
return function(**kwargs)
File "/mynedata/lib/api/apicalls.py", line 123, in login_user
res = result.get(timeout=5)
File "/usr/local/lib/python3.6/dist-packages/celery/result.py", line 224, in get
on_message=on_message,
File "/usr/local/lib/python3.6/dist-packages/celery/backends/async.py", line 188, in wait_for_pending
for _ in self._wait_for_pending(result, **kwargs):
File "/usr/local/lib/python3.6/dist-packages/celery/backends/async.py", line 259, in _wait_for_pending
raise TimeoutError('The operation timed out.')
celery.exceptions.TimeoutError: The operation timed out.
答案 0 :(得分：0)
问题不在于芹菜或RabbitMQ，而是完全不相关：
我使用os.subprocess.Popen(shlex.split(backend_cmd), stdout=subprocess.PIPE, stderr=subprocess.STDOUT)开始了芹菜工作者。原来那个子进程PIPE管道在某个点（我认为是2 ^ 16个字符）会满了，这时我的芹菜工人试图写入管道时卡住了，因此停止将结果写入结果后端。这意味着我看到的超时有效。
我不明白为什么time.status在超时后仍显示“ SUCCESS”。
答案 1 :(得分：0)
对我来说，设置正确的backend（而不是result_backend）可以解决此问题。
我这样设置：
app = Celery('tasks', broker=BROKER_URL, backend=BACKEND_URL)
此外，请确保消息代理正在运行。