Celery Beat - 工人消费消息,但从来没有Acks。他们

时间:2016-09-27 19:37:09

标签: python django celery

我有一个简单的Django网站,我想通过Celery / RabbitMQ添加一些预定的任务。我被卡住了,因为虽然节拍调度程序没有任何问题地抽出任务,但是工作人员却没有消耗它们。工人从未在RabbitMQ中将其标记为已确认。

这是来自ego.settings;

的Celery配置
from celery.schedules import crontab
...

# Celery configuration
BROKER_URL = os.environ.get('BROKER_URL', 'amqp://guest:guest@localhost:5672')
CELERY_RESULT_BACKEND = os.environ.get('CELERY_RESULT_BACKEND', 'disabled')
CELERY_ACCEPT_CONTENT = ['json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'

CELERYBEAT_SCHEDULE = {
    'account-notifications': {
        'task': 'ego.celery.account_alerts',
        'schedule': crontab(),
    },
}

我的芹菜入口点(ego.celery;

from __future__ import absolute_import

import os

from celery import Celery

# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'ego.settings')

from django.conf import settings

app = Celery('ego')

# Using a string here means the worker will not have to
# pickle the object when using Windows.
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)


@app.task
def account_alerts():
    print("nothing here, yet")

不,我没有忘记ego.__init__:)

from __future__ import absolute_import

# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .celery import app as celery_app

来自节拍的日志看起来很正常,但工人非常安静。

击败实例:

$ celery -A ego beat -s /tmp/celerybeat-schedule`
celery beat v3.1.23 (Cipater) is starting.
__    -    ... __   -        _
Configuration ->
    . broker -> amqp://guest:**@localhost:5672//
    . loader -> celery.loaders.app.AppLoader
    . scheduler -> celery.beat.PersistentScheduler
    . db -> /Users/pnovotnak/Documents/Cyrus/ego/ego/celerybeat-schedule
    . logfile -> [stderr]@%INFO
    . maxinterval -> now (0s)

[2016-09-27 19:14:07,463: INFO/MainProcess] beat: Starting...
[2016-09-27 19:14:07,486: INFO/MainProcess] Scheduler: Sending due task account-notifications (ego.celery.account_alerts)

工人实例:

$ celery --autoreload -A ego worker -l debug


[2016-09-27 19:14:59,626: DEBUG/MainProcess] | Worker: Preparing bootsteps.
[2016-09-27 19:14:59,630: DEBUG/MainProcess] | Worker: Building graph...
[2016-09-27 19:14:59,630: DEBUG/MainProcess] | Worker: New boot order: {Beat, Timer, Hub, Queues (intra), Pool, Autoreloader, StateDB, Autoscaler, Consumer}
[2016-09-27 19:14:59,645: DEBUG/MainProcess] | Consumer: Preparing bootsteps.
[2016-09-27 19:14:59,646: DEBUG/MainProcess] | Consumer: Building graph...
[2016-09-27 19:14:59,652: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Events, Heart, Mingle, Gossip, Agent, Tasks, Control, event loop}
[2016-09-27 19:14:59,655: DEBUG/MainProcess] | Worker: Starting Hub
[2016-09-27 19:14:59,655: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 19:14:59,655: DEBUG/MainProcess] | Worker: Starting Pool
[2016-09-27 19:14:59,655: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 19:14:59,655: DEBUG/MainProcess] | Worker: Starting Autoreloader
[2016-09-27 19:14:59,655: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 19:14:59,655: DEBUG/MainProcess] | Worker: Starting Consumer
[2016-09-27 19:14:59,655: DEBUG/MainProcess] | Consumer: Starting Connection
[2016-09-27 19:14:59,663: DEBUG/MainProcess] Start from server, version: 0.9, properties: {'product': 'RabbitMQ', 'platform': 'Erlang/OTP', 'version': '3.6.5', 'copyright': 'Copyright (C) 2007-2016 Pivotal Software, Inc.', 'information': 'Licensed under the MPL.  See http://www.rabbitmq.com/', 'cluster_name': 'rabbit@d78e61a6690d', 'capabilities': {'authentication_failure_close': True, 'exchange_exchange_bindings': True, 'per_consumer_qos': True, 'basic.nack': True, 'direct_reply_to': True, 'connection.blocked': True, 'consumer_priorities': True, 'publisher_confirms': True, 'consumer_cancel_notify': True}}, mechanisms: ['PLAIN', 'AMQPLAIN'], locales: ['en_US']
[2016-09-27 19:14:59,664: DEBUG/MainProcess] Open OK!
[2016-09-27 19:14:59,664: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2016-09-27 19:14:59,664: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 19:14:59,664: DEBUG/MainProcess] | Consumer: Starting Events
[2016-09-27 19:14:59,673: DEBUG/MainProcess] Start from server, version: 0.9, properties: {'product': 'RabbitMQ', 'platform': 'Erlang/OTP', 'version': '3.6.5', 'copyright': 'Copyright (C) 2007-2016 Pivotal Software, Inc.', 'information': 'Licensed under the MPL.  See http://www.rabbitmq.com/', 'cluster_name': 'rabbit@d78e61a6690d', 'capabilities': {'authentication_failure_close': True, 'exchange_exchange_bindings': True, 'per_consumer_qos': True, 'basic.nack': True, 'direct_reply_to': True, 'connection.blocked': True, 'consumer_priorities': True, 'publisher_confirms': True, 'consumer_cancel_notify': True}}, mechanisms: ['PLAIN', 'AMQPLAIN'], locales: ['en_US']
[2016-09-27 19:14:59,674: DEBUG/MainProcess] Open OK!
[2016-09-27 19:14:59,674: DEBUG/MainProcess] using channel_id: 1
[2016-09-27 19:14:59,675: DEBUG/MainProcess] Channel open
[2016-09-27 19:14:59,676: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 19:14:59,677: DEBUG/MainProcess] | Consumer: Starting Heart
[2016-09-27 19:14:59,678: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 19:14:59,679: DEBUG/MainProcess] | Consumer: Starting Mingle
[2016-09-27 19:14:59,679: INFO/MainProcess] mingle: searching for neighbors
[2016-09-27 19:14:59,679: DEBUG/MainProcess] using channel_id: 1
[2016-09-27 19:14:59,680: DEBUG/MainProcess] Channel open
[2016-09-27 19:15:00,694: INFO/MainProcess] mingle: all alone
[2016-09-27 19:15:00,695: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 19:15:00,695: DEBUG/MainProcess] | Consumer: Starting Gossip
[2016-09-27 19:15:00,695: DEBUG/MainProcess] using channel_id: 2
[2016-09-27 19:15:00,696: DEBUG/MainProcess] Channel open
[2016-09-27 19:15:00,700: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 19:15:00,701: DEBUG/MainProcess] | Consumer: Starting Tasks
[2016-09-27 19:15:00,704: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 19:15:00,704: DEBUG/MainProcess] | Consumer: Starting Control
[2016-09-27 19:15:00,704: DEBUG/MainProcess] using channel_id: 3
[2016-09-27 19:15:00,705: DEBUG/MainProcess] Channel open
[2016-09-27 19:15:00,708: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 19:15:00,708: DEBUG/MainProcess] | Consumer: Starting event loop
[2016-09-27 19:15:00,709: WARNING/MainProcess] celery@Slug.local ready.
[2016-09-27 19:15:00,709: DEBUG/MainProcess] | Worker: Hub.register Autoreloader...

这是RabbitMQ管理员视图;

rabbitmq admin interface screenshot

就是这样。之后什么都没有。必要的附加软件版本;

  • RabbitMQ 3.6.5(Erlang 19.0.7)
  • Python 3.5.2

最后,其中一个排队消息的内容;

{
    "args": [],
    "callbacks": null,
    "chord": null,
    "errbacks": null,
    "eta": null,
    "expires": null,
    "id": "c474852a-dd60-4027-959d-5a9436337b17",
    "kwargs": {},
    "retries": 0,
    "task": "ego.celery.account_alerts",
    "taskset": null,
    "timelimit": [
        null,
        null
    ],
    "utc": true
}

1 个答案:

答案 0 :(得分:0)

我找到了解决方案!这是--autoreload标志。不要使用它。

第二课:手工操作芹菜,而不是像我一样通过调用。这只会让日志输出做坏事让你的生活更加艰难。

似乎--autoreload标志导致工人因某种原因而中断。这是日志与现有/不存在标志的比较。

在以下输出中,我已将任务更改为return "nothing here, yet"而非print()

--autoreload;

$ celery --autoreload -A ego worker -l debug

[2016-09-27 20:43:15,106: DEBUG/MainProcess] | Worker: Preparing bootsteps.
[2016-09-27 20:43:15,109: DEBUG/MainProcess] | Worker: Building graph...
[2016-09-27 20:43:15,110: DEBUG/MainProcess] | Worker: New boot order: {Timer, Hub, Queues (intra), Pool, Autoscaler, Autoreloader, Beat, StateDB, Consumer}
[2016-09-27 20:43:15,115: DEBUG/MainProcess] | Consumer: Preparing bootsteps.
[2016-09-27 20:43:15,115: DEBUG/MainProcess] | Consumer: Building graph...
[2016-09-27 20:43:15,122: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Events, Mingle, Tasks, Control, Agent, Gossip, Heart, event loop}

 -------------- celery@Slug.local v3.1.23 (Cipater)
---- **** -----
--- * ***  * -- Darwin-16.0.0-x86_64-i386-64bit
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app:         ego:0x10486f748
- ** ---------- .> transport:   amqp://guest:**@localhost:5672//
- ** ---------- .> results:     disabled://
- *** --- * --- .> concurrency: 8 (prefork)
-- ******* ----
--- ***** ----- [queues]
 -------------- .> celery           exchange=celery(direct) key=celery


[tasks]
  . celery.backend_cleanup
  . celery.chain
  . celery.chord
  . celery.chord_unlock
  . celery.chunks
  . celery.group
  . celery.map
  . celery.starmap
  . ego.celery.account_alerts
  . ledger.tasks.AccountAlerts

[2016-09-27 20:43:15,125: DEBUG/MainProcess] | Worker: Starting Hub
[2016-09-27 20:43:15,125: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:43:15,125: DEBUG/MainProcess] | Worker: Starting Pool
[2016-09-27 20:43:15,409: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:43:15,410: DEBUG/MainProcess] | Worker: Starting Autoreloader
[2016-09-27 20:43:15,410: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:43:15,410: DEBUG/MainProcess] | Worker: Starting Consumer
[2016-09-27 20:43:15,410: DEBUG/MainProcess] | Consumer: Starting Connection
[2016-09-27 20:43:15,420: DEBUG/MainProcess] Start from server, version: 0.9, properties: {'capabilities': {'per_consumer_qos': True, 'direct_reply_to': True, 'basic.nack': True, 'consumer_priorities': True, 'publisher_confirms': True, 'authentication_failure_close': True, 'connection.blocked': True, 'consumer_cancel_notify': True, 'exchange_exchange_bindings': True}, 'information': 'Licensed under the MPL.  See http://www.rabbitmq.com/', 'version': '3.6.5', 'product': 'RabbitMQ', 'cluster_name': 'rabbit@d78e61a6690d', 'copyright': 'Copyright (C) 2007-2016 Pivotal Software, Inc.', 'platform': 'Erlang/OTP'}, mechanisms: ['PLAIN', 'AMQPLAIN'], locales: ['en_US']
[2016-09-27 20:43:15,422: DEBUG/MainProcess] Open OK!
[2016-09-27 20:43:15,422: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2016-09-27 20:43:15,422: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:43:15,422: DEBUG/MainProcess] | Consumer: Starting Events
[2016-09-27 20:43:15,429: DEBUG/MainProcess] Start from server, version: 0.9, properties: {'capabilities': {'per_consumer_qos': True, 'direct_reply_to': True, 'basic.nack': True, 'consumer_priorities': True, 'publisher_confirms': True, 'authentication_failure_close': True, 'connection.blocked': True, 'consumer_cancel_notify': True, 'exchange_exchange_bindings': True}, 'information': 'Licensed under the MPL.  See http://www.rabbitmq.com/', 'version': '3.6.5', 'product': 'RabbitMQ', 'cluster_name': 'rabbit@d78e61a6690d', 'copyright': 'Copyright (C) 2007-2016 Pivotal Software, Inc.', 'platform': 'Erlang/OTP'}, mechanisms: ['PLAIN', 'AMQPLAIN'], locales: ['en_US']
[2016-09-27 20:43:15,430: DEBUG/MainProcess] Open OK!
[2016-09-27 20:43:15,431: DEBUG/MainProcess] using channel_id: 1
[2016-09-27 20:43:15,433: DEBUG/MainProcess] Channel open
[2016-09-27 20:43:15,433: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:43:15,433: DEBUG/MainProcess] | Consumer: Starting Mingle
[2016-09-27 20:43:15,434: INFO/MainProcess] mingle: searching for neighbors
[2016-09-27 20:43:15,434: DEBUG/MainProcess] using channel_id: 1
[2016-09-27 20:43:15,435: DEBUG/MainProcess] Channel open
[2016-09-27 20:43:16,445: INFO/MainProcess] mingle: all alone
[2016-09-27 20:43:16,445: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:43:16,445: DEBUG/MainProcess] | Consumer: Starting Tasks
[2016-09-27 20:43:16,451: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:43:16,451: DEBUG/MainProcess] | Consumer: Starting Control
[2016-09-27 20:43:16,451: DEBUG/MainProcess] using channel_id: 2
[2016-09-27 20:43:16,452: DEBUG/MainProcess] Channel open
[2016-09-27 20:43:16,455: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:43:16,456: DEBUG/MainProcess] | Consumer: Starting Gossip
[2016-09-27 20:43:16,456: DEBUG/MainProcess] using channel_id: 3
[2016-09-27 20:43:16,456: DEBUG/MainProcess] Channel open
[2016-09-27 20:43:16,467: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:43:16,467: DEBUG/MainProcess] | Consumer: Starting Heart
[2016-09-27 20:43:16,468: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:43:16,469: DEBUG/MainProcess] | Consumer: Starting event loop
[2016-09-27 20:43:16,470: WARNING/MainProcess] celery@Slug.local ready.
[2016-09-27 20:43:16,470: DEBUG/MainProcess] | Worker: Hub.register Autoreloader...



^C
worker: Hitting Ctrl+C again will terminate all running tasks!

worker: Warm shutdown (MainProcess)
[2016-09-27 20:44:33,368: DEBUG/MainProcess] | Worker: Closing Hub...
[2016-09-27 20:44:33,368: DEBUG/MainProcess] | Worker: Closing Pool...
[2016-09-27 20:44:33,368: DEBUG/MainProcess] | Worker: Closing Autoreloader...
[2016-09-27 20:44:33,368: DEBUG/MainProcess] | Worker: Closing Consumer...
[2016-09-27 20:44:33,368: DEBUG/MainProcess] | Worker: Stopping Consumer...
[2016-09-27 20:44:33,368: DEBUG/MainProcess] | Consumer: Closing Connection...
[2016-09-27 20:44:33,368: DEBUG/MainProcess] | Consumer: Closing Events...
[2016-09-27 20:44:33,368: DEBUG/MainProcess] | Consumer: Closing Mingle...
[2016-09-27 20:44:33,368: DEBUG/MainProcess] | Consumer: Closing Tasks...
[2016-09-27 20:44:33,368: DEBUG/MainProcess] | Consumer: Closing Control...
[2016-09-27 20:44:33,368: DEBUG/MainProcess] | Consumer: Closing Gossip...
[2016-09-27 20:44:33,369: DEBUG/MainProcess] | Consumer: Closing Heart...
[2016-09-27 20:44:33,369: DEBUG/MainProcess] | Consumer: Closing event loop...
[2016-09-27 20:44:33,369: DEBUG/MainProcess] | Consumer: Stopping event loop...
[2016-09-27 20:44:33,369: DEBUG/MainProcess] | Consumer: Stopping Heart...
[2016-09-27 20:44:33,369: DEBUG/MainProcess] | Consumer: Stopping Gossip...
[2016-09-27 20:44:33,372: DEBUG/MainProcess] Closed channel #3
[2016-09-27 20:44:33,373: DEBUG/MainProcess] | Consumer: Stopping Control...
[2016-09-27 20:44:33,374: DEBUG/MainProcess] Closed channel #2
[2016-09-27 20:44:33,374: DEBUG/MainProcess] | Consumer: Stopping Tasks...
[2016-09-27 20:44:33,374: DEBUG/MainProcess] Canceling task consumer...
[2016-09-27 20:44:33,375: DEBUG/MainProcess] | Consumer: Stopping Mingle...
[2016-09-27 20:44:33,375: DEBUG/MainProcess] | Consumer: Stopping Events...
[2016-09-27 20:44:33,375: DEBUG/MainProcess] | Consumer: Stopping Connection...
[2016-09-27 20:44:33,375: DEBUG/MainProcess] | Worker: Stopping Autoreloader...
[2016-09-27 20:44:33,375: DEBUG/MainProcess] | Worker: Stopping Pool...
[2016-09-27 20:44:34,388: DEBUG/MainProcess] | Worker: Stopping Hub...
[2016-09-27 20:44:34,388: DEBUG/MainProcess] | Consumer: Shutdown Heart...
[2016-09-27 20:44:34,389: DEBUG/MainProcess] | Consumer: Shutdown Gossip...
[2016-09-27 20:44:34,389: DEBUG/MainProcess] | Consumer: Shutdown Control...
[2016-09-27 20:44:34,389: DEBUG/MainProcess] | Consumer: Shutdown Tasks...
[2016-09-27 20:44:34,389: DEBUG/MainProcess] Canceling task consumer...
[2016-09-27 20:44:34,389: DEBUG/MainProcess] Closing consumer channel...
[2016-09-27 20:44:34,389: DEBUG/MainProcess] | Consumer: Shutdown Events...
[2016-09-27 20:44:34,390: DEBUG/MainProcess] Closed channel #1
[2016-09-27 20:44:34,391: DEBUG/MainProcess] | Consumer: Shutdown Connection...
[2016-09-27 20:44:34,392: DEBUG/MainProcess] Closed channel #1
[2016-09-27 20:44:34,396: DEBUG/MainProcess] removing tasks from inqueue until task handler finished

没有--autoreload(注意前两个任务的“redelivered”状态);

$ celery -A ego worker -l debug

[2016-09-27 20:44:42,075: DEBUG/MainProcess] | Worker: Preparing bootsteps.
[2016-09-27 20:44:42,079: DEBUG/MainProcess] | Worker: Building graph...
[2016-09-27 20:44:42,080: DEBUG/MainProcess] | Worker: New boot order: {Timer, Hub, Queues (intra), Pool, Autoscaler, Autoreloader, Beat, StateDB, Consumer}
[2016-09-27 20:44:42,087: DEBUG/MainProcess] | Consumer: Preparing bootsteps.
[2016-09-27 20:44:42,088: DEBUG/MainProcess] | Consumer: Building graph...
[2016-09-27 20:44:42,101: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Agent, Events, Mingle, Gossip, Heart, Tasks, Control, event loop}

 -------------- celery@Slug.local v3.1.23 (Cipater)
---- **** -----
--- * ***  * -- Darwin-16.0.0-x86_64-i386-64bit
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app:         ego:0x10516f710
- ** ---------- .> transport:   amqp://guest:**@localhost:5672//
- ** ---------- .> results:     disabled://
- *** --- * --- .> concurrency: 8 (prefork)
-- ******* ----
--- ***** ----- [queues]
 -------------- .> celery           exchange=celery(direct) key=celery


[tasks]
  . celery.backend_cleanup
  . celery.chain
  . celery.chord
  . celery.chord_unlock
  . celery.chunks
  . celery.group
  . celery.map
  . celery.starmap
  . ego.celery.account_alerts
  . ledger.tasks.AccountAlerts

[2016-09-27 20:44:42,105: DEBUG/MainProcess] | Worker: Starting Hub
[2016-09-27 20:44:42,105: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:44:42,105: DEBUG/MainProcess] | Worker: Starting Pool
[2016-09-27 20:44:42,387: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:44:42,387: DEBUG/MainProcess] | Worker: Starting Consumer
[2016-09-27 20:44:42,388: DEBUG/MainProcess] | Consumer: Starting Connection
[2016-09-27 20:44:42,398: DEBUG/MainProcess] Start from server, version: 0.9, properties: {'copyright': 'Copyright (C) 2007-2016 Pivotal Software, Inc.', 'capabilities': {'publisher_confirms': True, 'authentication_failure_close': True, 'exchange_exchange_bindings': True, 'consumer_priorities': True, 'per_consumer_qos': True, 'consumer_cancel_notify': True, 'direct_reply_to': True, 'basic.nack': True, 'connection.blocked': True}, 'information': 'Licensed under the MPL.  See http://www.rabbitmq.com/', 'platform': 'Erlang/OTP', 'cluster_name': 'rabbit@d78e61a6690d', 'product': 'RabbitMQ', 'version': '3.6.5'}, mechanisms: ['PLAIN', 'AMQPLAIN'], locales: ['en_US']
[2016-09-27 20:44:42,399: DEBUG/MainProcess] Open OK!
[2016-09-27 20:44:42,399: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2016-09-27 20:44:42,400: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:44:42,400: DEBUG/MainProcess] | Consumer: Starting Events
[2016-09-27 20:44:42,408: DEBUG/MainProcess] Start from server, version: 0.9, properties: {'copyright': 'Copyright (C) 2007-2016 Pivotal Software, Inc.', 'capabilities': {'publisher_confirms': True, 'authentication_failure_close': True, 'exchange_exchange_bindings': True, 'consumer_priorities': True, 'per_consumer_qos': True, 'consumer_cancel_notify': True, 'direct_reply_to': True, 'basic.nack': True, 'connection.blocked': True}, 'information': 'Licensed under the MPL.  See http://www.rabbitmq.com/', 'platform': 'Erlang/OTP', 'cluster_name': 'rabbit@d78e61a6690d', 'product': 'RabbitMQ', 'version': '3.6.5'}, mechanisms: ['PLAIN', 'AMQPLAIN'], locales: ['en_US']
[2016-09-27 20:44:42,409: DEBUG/MainProcess] Open OK!
[2016-09-27 20:44:42,409: DEBUG/MainProcess] using channel_id: 1
[2016-09-27 20:44:42,412: DEBUG/MainProcess] Channel open
[2016-09-27 20:44:42,412: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:44:42,412: DEBUG/MainProcess] | Consumer: Starting Mingle
[2016-09-27 20:44:42,412: INFO/MainProcess] mingle: searching for neighbors
[2016-09-27 20:44:42,413: DEBUG/MainProcess] using channel_id: 1
[2016-09-27 20:44:42,426: DEBUG/MainProcess] Channel open
[2016-09-27 20:44:43,445: INFO/MainProcess] mingle: all alone
[2016-09-27 20:44:43,445: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:44:43,445: DEBUG/MainProcess] | Consumer: Starting Gossip
[2016-09-27 20:44:43,445: DEBUG/MainProcess] using channel_id: 2
[2016-09-27 20:44:43,446: DEBUG/MainProcess] Channel open
[2016-09-27 20:44:43,450: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:44:43,450: DEBUG/MainProcess] | Consumer: Starting Heart
[2016-09-27 20:44:43,451: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:44:43,451: DEBUG/MainProcess] | Consumer: Starting Tasks
[2016-09-27 20:44:43,456: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:44:43,456: DEBUG/MainProcess] | Consumer: Starting Control
[2016-09-27 20:44:43,456: DEBUG/MainProcess] using channel_id: 3
[2016-09-27 20:44:43,457: DEBUG/MainProcess] Channel open
[2016-09-27 20:44:43,460: DEBUG/MainProcess] ^-- substep ok
[2016-09-27 20:44:43,460: DEBUG/MainProcess] | Consumer: Starting event loop
[2016-09-27 20:44:43,461: WARNING/MainProcess] celery@Slug.local ready.
[2016-09-27 20:44:43,462: DEBUG/MainProcess] | Worker: Hub.register Pool...
[2016-09-27 20:44:43,462: DEBUG/MainProcess] basic.qos: prefetch_count->32
[2016-09-27 20:44:43,463: INFO/MainProcess] Received task: ego.celery.account_alerts[bc4c5b05-6306-4a57-b1a0-eca1e9282c13]
[2016-09-27 20:44:43,463: DEBUG/MainProcess] TaskPool: Apply <function _fast_trace_task at 0x105263950> (args:('ego.celery.account_alerts', 'bc4c5b05-6306-4a57-b1a0-eca1e9282c13', [], {}, {'is_eager': False, 'delivery_info': {'exchange': 'celery', 'routing_key': 'celery', 'redelivered': True, 'priority': 0}, 'correlation_id': 'bc4c5b05-6306-4a57-b1a0-eca1e9282c13', 'expires': None, 'chord': None, 'task': 'ego.celery.account_alerts', 'timelimit': [None, None], 'callbacks': None, 'headers': {}, 'utc': True, 'errbacks': None, 'eta': None, 'kwargs': {}, 'args': [], 'hostname': 'celery@Slug.local', 'taskset': None, 'group': None, 'retries': 0, 'id': 'bc4c5b05-6306-4a57-b1a0-eca1e9282c13', 'reply_to': '656c174e-c251-3e02-8132-df741c11a046'}) kwargs:{})
[2016-09-27 20:44:43,465: DEBUG/MainProcess] Task accepted: ego.celery.account_alerts[bc4c5b05-6306-4a57-b1a0-eca1e9282c13] pid:63322
[2016-09-27 20:44:43,478: INFO/MainProcess] Task ego.celery.account_alerts[bc4c5b05-6306-4a57-b1a0-eca1e9282c13] succeeded in 0.013773033046163619s: 'nothing here, yet'
[2016-09-27 20:44:45,458: INFO/MainProcess] Received task: ego.celery.account_alerts[b7673a47-dd76-4f1d-b414-03744fcfac22]
[2016-09-27 20:44:45,459: DEBUG/MainProcess] TaskPool: Apply <function _fast_trace_task at 0x105263950> (args:('ego.celery.account_alerts', 'b7673a47-dd76-4f1d-b414-03744fcfac22', [], {}, {'is_eager': False, 'delivery_info': {'exchange': 'celery', 'routing_key': 'celery', 'redelivered': True, 'priority': 0}, 'correlation_id': 'b7673a47-dd76-4f1d-b414-03744fcfac22', 'expires': None, 'chord': None, 'task': 'ego.celery.account_alerts', 'timelimit': [None, None], 'callbacks': None, 'headers': {}, 'utc': True, 'errbacks': None, 'eta': None, 'kwargs': {}, 'args': [], 'hostname': 'celery@Slug.local', 'taskset': None, 'group': None, 'retries': 0, 'id': 'b7673a47-dd76-4f1d-b414-03744fcfac22', 'reply_to': '656c174e-c251-3e02-8132-df741c11a046'}) kwargs:{})
[2016-09-27 20:44:45,461: DEBUG/MainProcess] Task accepted: ego.celery.account_alerts[b7673a47-dd76-4f1d-b414-03744fcfac22] pid:63327
[2016-09-27 20:44:45,462: INFO/MainProcess] Task ego.celery.account_alerts[b7673a47-dd76-4f1d-b414-03744fcfac22] succeeded in 0.0010689240298233926s: 'nothing here, yet'
[2016-09-27 20:45:00,002: INFO/MainProcess] Received task: ego.celery.account_alerts[2d464f3a-36f3-4f45-b303-aa477bc30545]
[2016-09-27 20:45:00,002: DEBUG/MainProcess] TaskPool: Apply <function _fast_trace_task at 0x105263950> (args:('ego.celery.account_alerts', '2d464f3a-36f3-4f45-b303-aa477bc30545', [], {}, {'is_eager': False, 'delivery_info': {'exchange': 'celery', 'routing_key': 'celery', 'redelivered': False, 'priority': 0}, 'correlation_id': '2d464f3a-36f3-4f45-b303-aa477bc30545', 'expires': None, 'chord': None, 'task': 'ego.celery.account_alerts', 'timelimit': [None, None], 'callbacks': None, 'headers': {}, 'utc': True, 'errbacks': None, 'eta': None, 'kwargs': {}, 'args': [], 'hostname': 'celery@Slug.local', 'taskset': None, 'group': None, 'retries': 0, 'id': '2d464f3a-36f3-4f45-b303-aa477bc30545', 'reply_to': '656c174e-c251-3e02-8132-df741c11a046'}) kwargs:{})
[2016-09-27 20:45:00,003: DEBUG/MainProcess] Task accepted: ego.celery.account_alerts[2d464f3a-36f3-4f45-b303-aa477bc30545] pid:63321
[2016-09-27 20:45:00,003: INFO/MainProcess] Task ego.celery.account_alerts[2d464f3a-36f3-4f45-b303-aa477bc30545] succeeded in 0.000625498010776937s: 'nothing here, yet'
[2016-09-27 20:46:00,002: INFO/MainProcess] Received task: ego.celery.account_alerts[f8eba8f2-1c48-48bd-a221-2676c735299e]
[2016-09-27 20:46:00,003: DEBUG/MainProcess] TaskPool: Apply <function _fast_trace_task at 0x105263950> (args:('ego.celery.account_alerts', 'f8eba8f2-1c48-48bd-a221-2676c735299e', [], {}, {'is_eager': False, 'delivery_info': {'exchange': 'celery', 'routing_key': 'celery', 'redelivered': False, 'priority': 0}, 'correlation_id': 'f8eba8f2-1c48-48bd-a221-2676c735299e', 'expires': None, 'chord': None, 'task': 'ego.celery.account_alerts', 'timelimit': [None, None], 'callbacks': None, 'headers': {}, 'utc': True, 'errbacks': None, 'eta': None, 'kwargs': {}, 'args': [], 'hostname': 'celery@Slug.local', 'taskset': None, 'group': None, 'retries': 0, 'id': 'f8eba8f2-1c48-48bd-a221-2676c735299e', 'reply_to': '656c174e-c251-3e02-8132-df741c11a046'}) kwargs:{})
[2016-09-27 20:46:00,004: DEBUG/MainProcess] Task accepted: ego.celery.account_alerts[f8eba8f2-1c48-48bd-a221-2676c735299e] pid:63325
[2016-09-27 20:46:00,005: INFO/MainProcess] Task ego.celery.account_alerts[f8eba8f2-1c48-48bd-a221-2676c735299e] succeeded in 0.0016709549818187952s: 'nothing here, yet'