RabbitMQ / Celery(事件):服务器意外关闭连接

时间:2019-06-29 19:04:15

标签: python rabbitmq celery eventlet

大约2-3分钟后,芹菜会引发以下回溯:

File "c:\program files (x86)\lib\site-packages\eventlet\hubs\selects.py", line 55, in wait
    listeners.get(fileno, noop).cb(fileno)
[2019-06-29 20:38:09,849: WARNING/MainProcess] File "c:\program files (x86)\lib\site-packages\celery\worker\pidbox.py", line 120, in loop
    connection.drain_events(timeout=1.0)
[2019-06-29 20:38:09,849: WARNING/MainProcess] File "c:\program files (x86)\lib\site-packages\kombu\connection.py", line 315, in drain_events
    return self.transport.drain_events(self.connection, **kwargs)
[2019-06-29 20:38:09,849: WARNING/MainProcess] File "c:\program files (x86)\lib\site-packages\kombu\transport\pyamqp.py", line 103, in drain_events
    return connection.drain_events(**kwargs)
[2019-06-29 20:38:09,849: WARNING/MainProcess] File "c:\program files (x86)\lib\site-packages\amqp\connection.py", line 500, in drain_events
    while not self.blocking_read(timeout):
[2019-06-29 20:38:09,849: WARNING/MainProcess] File "c:\program files (x86)\lib\site-packages\amqp\connection.py", line 505, in blocking_read
    frame = self.transport.read_frame()
[2019-06-29 20:38:09,849: WARNING/MainProcess] File "c:\program files (x86)\lib\site-packages\amqp\transport.py", line 252, in read_frame
    frame_header = read(7, True)
[2019-06-29 20:38:09,849: WARNING/MainProcess] File "c:\program files (x86)\lib\site-packages\amqp\transport.py", line 444, in _read
[2019-06-29 20:38:09,849: WARNING/MainProcess] OSError: Server unexpectedly closed connection
[2019-06-29 20:38:09,849: WARNING/MainProcess] Removing descriptor: 1044

尽管如此,当前任务的工作者仍在继续工作。但是在我的Web应用程序中,我一直在轮询URL以获取当前任务的更新。发生上述错误后,与RabbitMQ的连接似乎已关闭,因此应用程序不再能够访问结果后端(也包括RabbitMQ)。

我花了很多时间弄清楚到底是什么会引发该错误。我得出一个非常模糊的结论,那可能是因为工作程序(当前正在使用:eventlet,因为在Windows上运行)无法将心跳发送到RabbitMQ。 但是我变得非常困惑,因为有人建议将“心跳”值配置为0(正如我在下面的配置中所看到的那样)。但这也不能解决问题。

这是我当前的配置,这是关于github和stackoverflow的所有最佳建议的绝望杂技:

CELERY_BROKER_URL = 'pyamqp://'
CELERY_RESULT_BACKEND = 'amqp://'                           
CELERY_BROKER_TRANSPORT_OPTIONS = {'confirm_publish': True}                 
CELERY_BROKER_HEARTBEAT = 0
CELERY_BROKER_POOL_LIMIT = None
CELERY_BROKER_TRANSPORT_OPTIONS = {'confirm_publish': True}
CELERY_BROKER_CONNECTION_TIMEOUT = 20
CELERY_BROKER_CONNECTION_RETRY = True
CELERY_BROKER_CONNECTION_MAX_RETRIES = 100

CELERY_TIMEZONE = 'UTC'
CELERY_ENABLE_UTC = True
CELERY_IGNORE_RESULT = False
SQLALCHEMY_DATABASE_URI = userpass + basedir + dbname
SQLALCHEMY_TRACK_MODIFICATIONS = False

我希望RabbitMQ和Celery之间的连接是稳定的,并且不会发生连接错误。

目前对我来说最大的问题是,我完全不知道错误触发的确切原因。是工人没有向RabbitMQ发送心跳信号吗?是RabbitMQ本身吗?

每种提示我都很高兴。

编辑: 忘了提一下,我还读到这与主机名有关系。 我从芹菜开始:

  

celery -tasks.celery worker --loglevel = info --pool = eventlet --without-mingle --without-gossip --without-heartbeat

,它在localhost上运行(Web服务器通过XAMPP运行)。 如果我检查RabbitMQ控制台,它不是“ rabbit @ localhost”,而是“ rabbit @ DESKTOP-xxxx”。也许这会导致错误?

1 个答案:

答案 0 :(得分:1)

从所有CELERY_BROKER设置中删除CELERY_前缀。为我工作。

这是我celeryconfig.py中的一个片段:

BROKER_HEARTBEAT = 0

BROKER_POOL_LIMIT = None

BROKER_TRANSPORT_OPTIONS = {'confirm_publish': True}

BROKER_CONNECTION_TIMEOUT = 30

BROKER_CONNECTION_RETRY = True

BROKER_CONNECTION_MAX_RETRIES = 100

CELERY_RESULT_BACKEND = 'db+sqlite:///portal_tasks.db'

CELERY_TASK_TRACK_STARTED = True