我想知道,使用相同的代理拥有不同的Celery实例对象是一种好习惯吗?
目前,我有一个rabbitmq,作为3个Celery实例共享的单个代理。我的芹菜实例如下
insider_transaction
- 修正了计划工作人员。每分钟运行earning
- 由网络服务器创建的工作人员。stock_price
- 由网络服务器创建的工作人员。我设计的每个工人都在自己的docker容器中运行。我希望3名工人彼此独立。
然而,我意识到情况并非如此!
例如,earning
工作人员会错误地接收假设仅由stock_price
或insider_transaction
收到的邮件。
您将看到earning
工作人员收到此类消息。
earning_1 | The message has been ignored and discarded.
earning_1 |
earning_1 | Did you remember to import the module containing this task?
earning_1 | Or maybe you're using relative imports?
earning_1 |
earning_1 | Please see
earning_1 | http://docs.celeryq.org/en/latest/internals/protocol.html
earning_1 | for more information.
earning_1 |
earning_1 | The full contents of the message body was:
earning_1 | '[[], {}, {"callbacks": null, "errbacks": null, "chain": null, "chord": null}]' (77b)
earning_1 | Traceback (most recent call last):
earning_1 | File "/usr/local/lib/python3.6/site-packages/celery/worker/consumer/consumer.py", line 561, in on_task_received
earning_1 | strategy = strategies[type_]
earning_1 | KeyError: 'insider_transaction.run'
和这个
earning_1 | The message has been ignored and discarded.
earning_1 |
earning_1 | Did you remember to import the module containing this task?
earning_1 | Or maybe you're using relative imports?
earning_1 |
earning_1 | Please see
earning_1 | http://docs.celeryq.org/en/latest/internals/protocol.html
earning_1 | for more information.
earning_1 |
earning_1 | The full contents of the message body was:
earning_1 | '[[2, 3], {}, {"callbacks": null, "errbacks": null, "chain": null, "chord": null}]' (81b)
earning_1 | Traceback (most recent call last):
earning_1 | File "/usr/local/lib/python3.6/site-packages/celery/worker/consumer/consumer.py", line 561, in on_task_received
earning_1 | strategy = strategies[type_]
earning_1 | KeyError: 'stock_price.mul'
我不希望发生这种情况。在我的Web服务器端代码(Flask)中。我写了
celery0 = Celery('earning',
broker=CELERY_BROKER_URL,
backend=CELERY_RESULT_BACKEND)
celery1 = Celery('stock_price',
broker=CELERY_BROKER_URL,
backend=CELERY_RESULT_BACKEND)
@app.route('/do_work/<int:param1>/<int:param2>')
def do_work(param1,param2):
task0 = celery0.send_task('earning.add', args=[param1, param2], kwargs={})
task1 = celery1.send_task('stock_price.mul', args=[param1, param2], kwargs={})
因此,我希望earning
工作人员只会收到earning
条消息,而不会收到stock_price
或insider_transaction
条消息。
我可以知道,为什么会出现这个问题?不同的Celery实例是否不可能共享单个经纪人?
可以从https://github.com/yccheok/celery-hello-world
结帐一个演示此问题的项目docker-compose build
docker-compose up -d
http://localhost:5000/do_work/2/3
docker-compose up earning