Celery在日志中出现SQS错误

时间:2013-06-23 16:45:34

标签: django django-celery amazon-sqs

我正在将Celery与Amazon SQS一起使用,我在日志中看到了大量这些内容:

[2013-06-19 19:03:42,890: ERROR/MainProcess] Pool callback raised exception: KeyError('8a8fe51c-b69f-498e-95c1-f037e2f2d4a8',)
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/billiard/pool.py", line 129, in safe_apply_callback
    fun(*args)
  File "/usr/local/lib/python2.7/dist-packages/celery/worker/job.py", line 301, in on_accepted
    self.acknowledge()
  File "/usr/local/lib/python2.7/dist-packages/celery/worker/job.py", line 439, in acknowledge
    self.on_ack(logger, self.connection_errors)
  File "/usr/local/lib/python2.7/dist-packages/kombu/transport/base.py", line 98, in ack_log_error
    self.ack()
  File "/usr/local/lib/python2.7/dist-packages/kombu/transport/base.py", line 93, in ack
    self.channel.basic_ack(self.delivery_tag)
  File "/usr/local/lib/python2.7/dist-packages/kombu/transport/SQS.py", line 252, in basic_ack
    delivery_info = self.qos.get(delivery_tag).delivery_info
  File "/usr/local/lib/python2.7/dist-packages/kombu/transport/virtual/__init__.py", line 127, in get
    return self._delivered[delivery_tag]
KeyError: '8a8fe51c-b69f-498e-95c1-f037e2f2d4a8'

我试图谷歌这个,似乎没有人真正知道它来自哪里?有谁知道最近发生了什么?我注意到一些奇怪的行为,任务失败,然后它一次又一次地重试相同的任务。过了一段时间,它不仅会重试,而且甚至不会在日志中显示错误(或任何日志声明)。它只是说它有一个任务,它会尝试它。我不确定发生了什么事?

我只是使用以下方法进行清除:

from celery.task.control import discard_all
discard_all()

以某种方式,任务又重新开始了!有没有人知道这有甚么可能?

1 个答案:

答案 0 :(得分:1)

这是由Kombu的SQS后端中的错误引起的,并已在https://github.com/celery/kombu/pull/431中修复