Question

我在运行Celery时收到AssertionError。这个问题具有持久性和可重复性，因为如果我运行一个约700,000个子任务的工作，问题就会在某个时候完成。但是，如果我只运行180个子任务，则不会发生这种情况。

[2014-06-10 10:27:51,355: ERROR/MainProcess] Unrecoverable error: AssertionError()
Traceback (most recent call last):
  File "/Users/beccapetrin/Documents/Development/automated_reports/lib/python2.7/site-packages/celery/worker/__init__.py", line 206, in start
self.blueprint.start(self)
  File "/Users/beccapetrin/Documents/Development/automated_reports/lib/python2.7/site-packages/celery/bootsteps.py", line 123, in start
step.start(parent)
  File "/Users/beccapetrin/Documents/Development/automated_reports/lib/python2.7/site-packages/celery/bootsteps.py", line 373, in start
return self.obj.start()
  File "/Users/beccapetrin/Documents/Development/automated_reports/lib/python2.7/site-packages/celery/worker/consumer.py", line 278, in start
blueprint.start(self)
  File "/Users/beccapetrin/Documents/Development/automated_reports/lib/python2.7/site-packages/celery/bootsteps.py", line 123, in start
step.start(parent)
  File "/Users/beccapetrin/Documents/Development/automated_reports/lib/python2.7/site-packages/celery/worker/consumer.py", line 821, in start
c.loop(*c.loop_args())
  File "/Users/beccapetrin/Documents/Development/automated_reports/lib/python2.7/site-packages/celery/worker/loops.py", line 72, in asynloop
next(loop)
  File "/Users/beccapetrin/Documents/Development/automated_reports/lib/python2.7/site-packages/kombu/async/hub.py", line 324, in create_loop
cb(*cbargs)
  File "/Users/beccapetrin/Documents/Development/automated_reports/lib/python2.7/site-packages/celery/concurrency/asynpool.py", line 263, in on_result_readable
next(it)
  File "/Users/beccapetrin/Documents/Development/automated_reports/lib/python2.7/site-packages/celery/concurrency/asynpool.py", line 200, in _recv_message
assert not isblocking(fd)
AssertionError

以下是我的配置详情：

$ celery report

software -> celery:3.1.12 (Cipater) kombu:3.0.19 py:2.7.5
            billiard:3.3.0.17 py-amqp:1.4.5
platform -> system:Darwin arch:64bit imp:CPython
loader   -> celery.loaders.default.Loader
settings -> transport:amqp results:amqp://username:password@localhost:5672

CELERY_QUEUES:
    (<unbound Queue automated_reports -> <unbound Exchange default(direct)> -> automated_reports>,)
CELERY_DEFAULT_ROUTING_KEY: '********'
CELERY_IMPORTS:
    ('automated_reports.queue.tasks', 'automated_reports.queue.subtasks')
CELERY_RESULT_PERSISTENT: True
CELERY_ROUTES: {
    'automated_reports.queue.tasks.run_device_info_report': {   'queue': 'automated_reports'},
    'uploader.queue.subtasks.multithread_device_listing': {   'queue': 'automated_reports'},
    'uploader.queue.subtasks.multithread_individual_device': {   'queue': 'automated_reports'},
    'uploader.queue.tasks.multithread_device_listing': {   'queue': 'automated_reports'},
    'uploader.queue.tasks.multithread_individual_device': {   'queue': 'automated_reports'}}
CELERY_DEFAULT_QUEUE: 'automated_reports'
BROKER_URL: 'amqp://guest:********@localhost:5672//'
CELERY_RESULT_BACKEND: 'amqp://username:password@localhost:5672'

我的工作人员启动命令是：

~/Documents/Development/automated_reports/bin/celeryd --loglevel=DEBUG --autoreload -A automated_reports.queue.tasks -Q automated_reports -B --schedule=~/Documents/Development/automated_reports/log/celerybeat --autoscale=10,3 -Ofair

我将我的应用程序作为一项任务运行，然后使用组合在一起的芹菜子任务启动子任务。

操作系统版本为Mac OS X，版本10.9.3。

这看起来与我在Github上看到的一些Celery问题非常相似，但并不完全相同。谢谢！

根据与芹菜团队的对话，我也在Github上发布了这个问题：https://github.com/celery/celery/issues/2086

Answer 1

当我从worker启动命令中删除了--autoscale标志时，这解决了。

Celery AssertionError

1 个答案: