Question

我尝试使用celery=4.2.0上的Python=3.6，Rabbitmq=3.7.9和windows 10执行简单的任务，该任务已由工作人员接收，但未执行且未得到确认

任务描述：

@app.task()
def simpleTask(*args):
    logging.info('{}'.format(args[0]))

任务执行命令：

simpleTask.s('1').apply_async()

我尝试使用-P gevent(version=1.4.0)和eventlet(version=0.24.1)运行该应用程序，但没有成功。尽管在使用solo选项时任务可以正确执行，但对于我的最终目标，由工作者在单线程环境中运行任务与我的要求不符。

Celery worker命令：

argv = [
        'worker',
        '-l=DEBUG',
        '-P=eventlet',
        '--without-gossip',
        '--without-mingle',
        '--without-heartbeat',
        '-Ofair',
    ]
app.worker_main(argv)

输出：

[2019-03-10 23:08:18,612: DEBUG/MainProcess] | Worker: Preparing bootsteps.
[2019-03-10 23:08:18,618: DEBUG/MainProcess] | Worker: Building graph...
[2019-03-10 23:08:18,619: DEBUG/MainProcess] | Worker: New boot order: {StateDB, Timer, Hub, Pool, Autoscaler, Beat, Consumer}
[2019-03-10 23:08:18,938: DEBUG/MainProcess] | Consumer: Preparing bootsteps.
[2019-03-10 23:08:18,939: DEBUG/MainProcess] | Consumer: Building graph...
[2019-03-10 23:08:18,951: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Agent, Events, Heart, Mingle, Gossip, Tasks, Control, event loop}

[2019-03-10 23:08:18,962: DEBUG/MainProcess] | Worker: Starting Pool
 -------------- celery@DESKTOP-IEMN7E0 v4.2.1 (windowlicker)
[2019-03-10 23:08:18,962: DEBUG/MainProcess] ^-- substep ok
---- **** ----- 
[2019-03-10 23:08:18,962: DEBUG/MainProcess] | Worker: Starting Consumer
--- * ***  * -- Windows-10-10.0.17134-SP0 2019-03-10 23:08:18
[2019-03-10 23:08:18,962: DEBUG/MainProcess] | Consumer: Starting Connection
-- * - **** --- 
- ** ---------- [config]
- ** ---------- .> app:         workflowEngine.Main.Server.tasks:0x2771cf629e8
- ** ---------- .> transport:   amqp://guest:**@localhost:5672//
- ** ---------- .> results:     mongodb://
- *** --- * --- .> concurrency: 4 (eventlet)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** ----- 
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery


[tasks]
  . PanelTask
  . celery.accumulate
  . celery.backend_cleanup
  . celery.chain
  . celery.chord
  . celery.chord_unlock
  . celery.chunks
  . celery.group
  . celery.map
  . celery.starmap
  . workflowEngine.Main.Server.tasks.finalChordTask
  . workflowEngine.Main.Server.tasks.simpleTask

[2019-03-10 23:08:18,978: DEBUG/MainProcess] Start from server, version: 0.9, properties: {'capabilities': {'publisher_confirms': True, 'exchange_exchange_bindings': True, 'basic.nack': True, 'consumer_cancel_notify': True, 'connection.blocked': True, 'consumer_priorities': True, 'authentication_failure_close': True, 'per_consumer_qos': True, 'direct_reply_to': True}, 'cluster_name': 'rabbit@DESKTOP-IEMN7E0', 'copyright': 'Copyright (C) 2007-2018 Pivotal Software, Inc.', 'information': 'Licensed under the MPL.  See http://www.rabbitmq.com/', 'platform': 'Erlang/OTP 21.1', 'product': 'RabbitMQ', 'version': '3.7.9'}, mechanisms: [b'AMQPLAIN', b'PLAIN'], locales: ['en_US']
[2019-03-10 23:08:18,980: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2019-03-10 23:08:18,981: DEBUG/MainProcess] ^-- substep ok
[2019-03-10 23:08:18,981: DEBUG/MainProcess] | Consumer: Starting Events
[2019-03-10 23:08:18,998: DEBUG/MainProcess] Start from server, version: 0.9, properties: {'capabilities': {'publisher_confirms': True, 'exchange_exchange_bindings': True, 'basic.nack': True, 'consumer_cancel_notify': True, 'connection.blocked': True, 'consumer_priorities': True, 'authentication_failure_close': True, 'per_consumer_qos': True, 'direct_reply_to': True}, 'cluster_name': 'rabbit@DESKTOP-IEMN7E0', 'copyright': 'Copyright (C) 2007-2018 Pivotal Software, Inc.', 'information': 'Licensed under the MPL.  See http://www.rabbitmq.com/', 'platform': 'Erlang/OTP 21.1', 'product': 'RabbitMQ', 'version': '3.7.9'}, mechanisms: [b'AMQPLAIN', b'PLAIN'], locales: ['en_US']
[2019-03-10 23:08:19,001: DEBUG/MainProcess] ^-- substep ok
[2019-03-10 23:08:19,001: DEBUG/MainProcess] | Consumer: Starting Tasks
[2019-03-10 23:08:19,010: DEBUG/MainProcess] using channel_id: 1
[2019-03-10 23:08:19,028: DEBUG/MainProcess] Channel open
[2019-03-10 23:08:19,037: DEBUG/MainProcess] ^-- substep ok
[2019-03-10 23:08:19,037: DEBUG/MainProcess] | Consumer: Starting Control
[2019-03-10 23:08:19,052: DEBUG/MainProcess] ^-- substep ok
[2019-03-10 23:08:19,052: DEBUG/MainProcess] | Consumer: Starting event loop
[2019-03-10 23:08:19,054: INFO/MainProcess] celery@DESKTOP-IEMN7E0 ready.
[2019-03-10 23:08:19,055: DEBUG/MainProcess] basic.qos: prefetch_count->16
[2019-03-10 23:09:48,701: INFO/MainProcess] Received task: workflowEngine.Main.Server.tasks.simpleTask[13644976-55e4-4df4-a48a-a9274ff3daff]  
[2019-03-10 23:09:48,703: DEBUG/MainProcess] TaskPool: Apply <function _fast_trace_task at 0x000002771E1B9C80> (args:('workflowEngine.Main.Server.tasks.simpleTask', '13644976-55e4-4df4-a48a-a9274ff3daff', {'lang': 'py', 'task': 'workflowEngine.Main.Server.tasks.simpleTask', 'id': '13644976-55e4-4df4-a48a-a9274ff3daff', 'shadow': None, 'eta': None, 'expires': None, 'group': None, 'retries': 0, 'timelimit': [None, None], 'root_id': '13644976-55e4-4df4-a48a-a9274ff3daff', 'parent_id': None, 'argsrepr': "('1',)", 'kwargsrepr': '{}', 'origin': 'gen16440@DESKTOP-IEMN7E0', 'reply_to': 'e1566cdc-a4b7-3017-8167-36ea0a95af14', 'correlation_id': '13644976-55e4-4df4-a48a-a9274ff3daff', 'delivery_info': {'exchange': '', 'routing_key': 'celery', 'priority': 0, 'redelivered': False}}, '[["1"], {}, {"callbacks": null, "errbacks": null, "chain": null, "chord": null}]', 'application/json', 'utf-8') kwargs:{})

Answer 1

在进行一些调试之后，我发现我的问题传递了-P参数错误，这是令人惊讶的，原因是通过celery输出，该应用程序确实识别出我想使用eventlet软件包名称。

argv = [
    'worker',
    '-l=DEBUG',
    '-P',
    'eventlet',
    '-c=100',
    '--without-gossip',
    '--without-mingle',
    '--without-heartbeat',
    '-Ofair',
]
app.worker_main(argv)

此命令对eventlet和gevent来说对我来说很好

芹菜工人收到任务但未执行

1 个答案: