Question

当任务的负载增加并且由于周期性而再次执行任务时，周期性的芹菜任务会卡住。

为了产生负载，我在test_task1中增加了睡眠，结果没有执行所有队列的任务，这是我仅在屏幕上看到的日志：

[2018-06-19 11:52:49,939 > beat:239 INFO]:Scheduler: Sending due task test_task1 (test_celery_tasks.test_task1)
[2018-06-19 11:52:49,945 > beat:239 INFO]:Scheduler: Sending due task test_task2 (test_celery_tasks.test_task2)
[2018-06-19 11:52:49,951 > beat:239 INFO]:Scheduler: Sending due task test_task3 (test_celery_tasks.test_task3

计划程序：

app = Celery('my_scheduler3')
app.conf.update(
CELERYD_LOG_FORMAT='[%(asctime)s > %(module)s:%(lineno)d %(levelname)s]:%(message)s',
BROKER_URL='redis://localhost:6379/0',
BROKER_TRANSPORT = 'redis',
CELERY_RESULT_BACKEND = 'redis://localhost:6379/0',
CELERY_TASK_SERIALIZER='json',
CELERY_ACCEPT_CONTENT=['json'],
CELERYBEAT_SCHEDULE={
    'test_task1':{
        'task':'test_celery_tasks.test_task1',
        'schedule': 2.0,
        'options': {'queue' : 'Q-1'},

    },

任务：

@app.task(name='test_celery_tasks.test_task1')
def test_task1():
    try:
        print('Before SLEEP')
        time.sleep(60)
        print('After SLEEP')

    except Exception as e:
        logging.error("Error [{0}] at line [{1}]".format(
            str(e), sys.exc_info()[2].tb_lineno)

Cmd运行调度程序：

~/venv/bin/celery worker -A test_celery_scheduler --loglevel=info -n worker__1 --beat -Q Q-1,Q-2,Q-3 --purge  -s ~/test_scheduler

发送到期任务-Celery Python2.7

0 个答案: