Question

当MainProcess失去与代理的连接时，我们一直遇到Celery的奇怪问题。 Celery会在下面生成错误，然后开始使用100％CPU。工人们仍将继续工作。我可以看到，RabbitMQ认为连接是超时的。我们在更新到芹菜3后经常收到这些错误。

我觉得它与非阻塞消息有关，但我并没有真正理解代码。

有没有办法提前检测这些或阻止芹菜使用100％CPU？

芹菜：3.0.4
AMQP：1.0.10
RabbitMQ：2.8.4

时间戳相隔2小时，因为RabbitMQ报告GMT和芹菜当地时间。

芹菜错误

[2013-05-09 18:20:20,204: ERROR/MainProcess] Consumer: Connection to broker lost. Trying to re-establish the connection...
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/celery/worker/consumer.py", line 369, in start
    self.consume_messages()
  File "/usr/local/lib/python2.7/site-packages/celery/worker/consumer.py", line 450, in consume_messages
    readers[fileno](fileno, event)
  File "/usr/local/lib/python2.7/site-packages/kombu/connection.py", line 290, in drain_nowait
    self.drain_events(timeout=0)
  File "/usr/local/lib/python2.7/site-packages/kombu/connection.py", line 279, in drain_events
    return self.transport.drain_events(self.connection, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 91, in drain_events
    return connection.drain_events(**kwargs)
  File "/usr/local/lib/python2.7/site-packages/amqp/connection.py", line 266, in drain_events
    chanmap, None, timeout=timeout,
  File "/usr/local/lib/python2.7/site-packages/amqp/connection.py", line 328, in _wait_multiple
    channel, method_sig, args, content = read_timeout(timeout)
  File "/usr/local/lib/python2.7/site-packages/amqp/connection.py", line 299, in read_timeout
    return self.method_reader.read_method()
  File "/usr/local/lib/python2.7/site-packages/amqp/method_framing.py", line 187, in read_method
    raise m
IOError: Socket closed

RabbitMQ错误

=ERROR REPORT==== 9-May-2013::16:20:20 ===
closing AMQP connection <0.1813.0> (192.168.201.104:12809 -> 192.168.201.104:5672):
{timeout,running}

Answer 1

我关闭了经纪人听到的声音，这似乎已经解决了问题。但是，我不太确定，因为我没有办法重现这个错误。

Celery连接代理丢失使CPU使用率达到100％

1 个答案: