Question

我开始使用Celery。

Y执行shell（virt_env激活）命令：python manage.py celeryd --verbosity = 2 --loglevel = DEBUG

似乎是开始连接但失败但错误...

我有：操作系统：Debian 6 Python 2.6 Django 1.6.1

我的PIP冻结：

Django==1.6.1
MySQL-python==1.2.4b4
PIL==1.1.7
South==0.8.4
amqp==1.4.5
amqplib==1.0.2
anyjson==0.3.3
billiard==3.3.0.17
celery==3.1.11
defusedxml==0.4.1
django-bakery==0.2.2
django-celery==3.1.10
django-debug-toolbar==1.1
django-debug-toolbar-template-timings==0.6.1
django-password-reset==0.6
django-social-auth==0.7.28
djangorestframework==2.3.13
gdata==2.0.18
hachoir-core==1.3.3
hachoir-metadata==1.3.3
hachoir-parser==1.3.4
httplib2==0.8
importlib==1.0.3
kombu==3.0.16
mutagen==1.22
oauth2==1.5.211
oauthlib==0.6.1
ordereddict==1.1
python-memcached==1.53
python-openid==2.2.5
python-twitter==1.3.1
pytz==2014.2
rarfile==2.6
requests==2.1.0
requests-oauthlib==0.4.0
simplejson==3.4.1
sorl-thumbnail==11.12
sqlparse==0.1.11
twython==3.1.2
wsgiref==0.1.2

我的追溯：

[2014-05-14 21:20:09,426: DEBUG/MainProcess] | Worker: Preparing bootsteps.
[2014-05-14 21:20:09,430: DEBUG/MainProcess] | Worker: Building graph...
[2014-05-14 21:20:09,431: DEBUG/MainProcess] | Worker: New boot order: {StateDB, Timer, Hub, Queues (intra), Pool, Autoscaler, Autoreloader, Beat, Consumer}
[2014-05-14 21:20:09,439: DEBUG/MainProcess] | Consumer: Preparing bootsteps.
[2014-05-14 21:20:09,439: DEBUG/MainProcess] | Consumer: Building graph...
[2014-05-14 21:20:09,447: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Events, Mingle, Gossip, Tasks, Control, Heart, Agent, event loop}
[2014-05-14 21:20:09,448: WARNING/MainProcess] /home/federico/overactive/lib/python2.6/site-packages/celery/apps/worker.py:161: CDeprecationWarning: 
Starting from version 3.2 Celery will refuse to accept pickle by default.

The pickle serializer is a security concern as it may give attackers
the ability to execute any command. It's important to secure
your broker from unauthorized access when using pickle, so we think
that enabling pickle should require a deliberate action and not be
the default choice.

If you depend on pickle then you should set a setting to disable this
warning and to be sure that everything will continue working
when you upgrade to Celery 3.2::

CELERY_ACCEPT_CONTENT = ['pickle', 'json', 'msgpack', 'yaml']
You must only enable the serializers that you will actually use.

warnings.warn(CDeprecationWarning(W_PICKLE_DEPRECATED))

-------------- celery@debian v3.1.11 (Cipater)
---- **** ----- 
--- * *** * -- Linux-2.6.32-5-amd64-x86_64-with-debian-6.0.9
-- * - **** ---

** ---------- [config]
** ---------- .> app: default:0x2acb410 (.default.Loader)
** ---------- .> transport: amqp://guest@localhost:5672//
** ---------- .> results: disabled
*** --- * --- .> concurrency: 1 (prefork) -- ******* ---- --- ***** ----- [queues] -------------- .> celery exchange=celery(direct) key=celery
[tasks]
. celery.backend_cleanup
. celery.chain
. celery.chord
. celery.chord_unlock
. celery.chunks
. celery.group
. celery.map
. celery.starmap

[2014-05-14 21:20:09,451: DEBUG/MainProcess] | Worker: Starting Hub
[2014-05-14 21:20:09,452: DEBUG/MainProcess] ^-- substep ok
[2014-05-14 21:20:09,452: DEBUG/MainProcess] | Worker: Starting Pool
[2014-05-14 21:20:09,464: DEBUG/MainProcess] ^-- substep ok
[2014-05-14 21:20:09,478: DEBUG/MainProcess] | Worker: Starting Consumer
[2014-05-14 21:20:09,479: DEBUG/MainProcess] | Consumer: Starting Connection
[2014-05-14 21:20:09,525: DEBUG/MainProcess] Start from server, version: 8.0, properties: {u'platform': u'Erlang/OTP', u'product': u'RabbitMQ', u'version': u'1.8.1', u'copyright': u'Copyright (C) 2007-2010 LShift Ltd., Cohesive Financial Technologies LLC., and Rabbit Technologies Ltd.', u'information': u'Licensed under the MPL. See http://www.rabbitmq.com/'}, mechanisms: [u'PLAIN', u'AMQPLAIN'], locales: [u'en_US']
[2014-05-14 21:20:09,526: DEBUG/MainProcess] Open OK!
[2014-05-14 21:20:09,527: INFO/MainProcess] Connected to amqp://guest@127.0.0.1:5672//
[2014-05-14 21:20:09,527: DEBUG/MainProcess] ^-- substep ok
[2014-05-14 21:20:09,531: DEBUG/MainProcess] | Consumer: Starting Events
[2014-05-14 21:20:09,584: DEBUG/MainProcess] Start from server, version: 8.0, properties: {u'platform': u'Erlang/OTP', u'product': u'RabbitMQ', u'version': u'1.8.1', u'copyright': u'Copyright (C) 2007-2010 LShift Ltd., Cohesive Financial Technologies LLC., and Rabbit Technologies Ltd.', u'information': u'Licensed under the MPL. See http://www.rabbitmq.com/'}, mechanisms: [u'PLAIN', u'AMQPLAIN'], locales: [u'en_US']
[2014-05-14 21:20:09,586: DEBUG/MainProcess] Open OK!
[2014-05-14 21:20:09,586: DEBUG/MainProcess] using channel_id: 1
[2014-05-14 21:20:09,587: DEBUG/MainProcess] Channel open
[2014-05-14 21:20:09,587: DEBUG/MainProcess] ^-- substep ok
[2014-05-14 21:20:09,588: DEBUG/MainProcess] | Consumer: Starting Mingle
[2014-05-14 21:20:09,588: INFO/MainProcess] mingle: searching for neighbors
[2014-05-14 21:20:09,589: DEBUG/MainProcess] using channel_id: 1
[2014-05-14 21:20:09,594: DEBUG/MainProcess] Channel open
[2014-05-14 21:20:09,610: ERROR/MainProcess] Unrecoverable error: UnicodeDecodeError('utf8', '\xf5', 0, 1, 'invalid start byte')
Traceback (most recent call last):
File "/home/federico/overactive/lib/python2.6/site-packages/celery/worker/init.py", line 206, in start
self.blueprint.start(self)
File "/home/federico/overactive/lib/python2.6/site-packages/celery/bootsteps.py", line 123, in start
step.start(parent)
File "/home/federico/overactive/lib/python2.6/site-packages/celery/bootsteps.py", line 373, in start
return self.obj.start()
File "/home/federico/overactive/lib/python2.6/site-packages/celery/worker/consumer.py", line 278, in start
blueprint.start(self)
File "/home/federico/overactive/lib/python2.6/site-packages/celery/bootsteps.py", line 123, in start
step.start(parent)
File "/home/federico/overactive/lib/python2.6/site-packages/celery/worker/consumer.py", line 569, in start
replies = I.hello(c.hostname, revoked.data) or {}
File "/home/federico/overactive/lib/python2.6/site-packages/celery/app/control.py", line 111, in hello
return self.request('hello', from_node=from_node, revoked=revoked)
File "/home/federico/overactive/lib/python2.6/site-packages/celery/app/control.py", line 70, in _request
timeout=self.timeout, reply=True,
File "/home/federico/overactive/lib/python2.6/site-packages/celery/app/control.py", line 306, in broadcast
limit, callback, channel=channel,
File "/home/federico/overactive/lib/python2.6/site-packages/kombu/pidbox.py", line 300, in _broadcast
channel=chan)
File "/home/federico/overactive/lib/python2.6/site-packages/kombu/pidbox.py", line 336, in _collect
with consumer:
File "/home/federico/overactive/lib/python2.6/site-packages/kombu/messaging.py", line 396, in __enter
self.consume()
File "/home/federico/overactive/lib/python2.6/site-packages/kombu/messaging.py", line 445, in consume
self.basic_consume(T, no_ack=no_ack, nowait=False)
File "/home/federico/overactive/lib/python2.6/site-packages/kombu/messaging.py", line 567, in basic_consume
no_ack=no_ack, nowait=nowait)
File "/home/federico/overactive/lib/python2.6/site-packages/kombu/entity.py", line 611, in consume
nowait=nowait)
File "/home/federico/overactive/lib/python2.6/site-packages/amqp/channel.py", line 1787, in basic_consume
(60, 21), # Channel.basic_consume_ok
File "/home/federico/overactive/lib/python2.6/site-packages/amqp/abstract_channel.py", line 67, in wait
self.channel_id, allowed_methods)
File "/home/federico/overactive/lib/python2.6/site-packages/amqp/connection.py", line 270, in _wait_method
self.wait()
File "/home/federico/overactive/lib/python2.6/site-packages/amqp/abstract_channel.py", line 69, in wait
return self.dispatch_method(method_sig, args, content)
File "/home/federico/overactive/lib/python2.6/site-packages/amqp/abstract_channel.py", line 87, in dispatch_method
return amqp_method(self, args)
File "/home/federico/overactive/lib/python2.6/site-packages/amqp/connection.py", line 530, in _blocked
reason = args.read_shortstr()
File "/home/federico/overactive/lib/python2.6/site-packages/amqp/serialization.py", line 120, in read_shortstr
return self.input.read(slen).decode('utf-8')
File "/home/federico/overactive/lib/python2.6/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xf5 in position 0: invalid start byte
[2014-05-14 21:20:09,629: DEBUG/MainProcess] | Worker: Closing Hub...
[2014-05-14 21:20:09,629: DEBUG/MainProcess] | Worker: Closing Pool...
[2014-05-14 21:20:09,629: DEBUG/MainProcess] | Worker: Closing Consumer...
[2014-05-14 21:20:09,630: DEBUG/MainProcess] | Worker: Stopping Consumer...
[2014-05-14 21:20:09,630: DEBUG/MainProcess] | Worker: Stopping Pool...
[2014-05-14 21:20:10,642: DEBUG/MainProcess] | Worker: Stopping Hub...
[2014-05-14 21:20:10,642: DEBUG/MainProcess] | Consumer: Shutdown Heart...
[2014-05-14 21:20:10,642: DEBUG/MainProcess] | Consumer: Shutdown Control...
[2014-05-14 21:20:10,643: DEBUG/MainProcess] | Consumer: Shutdown Tasks...
[2014-05-14 21:20:10,643: DEBUG/MainProcess] | Consumer: Shutdown Gossip...
[2014-05-14 21:20:10,643: DEBUG/MainProcess] | Consumer: Shutdown Events...
[2014-05-14 21:20:10,644: DEBUG/MainProcess] Closed channel #1
[2014-05-14 21:20:10,645: WARNING/MainProcess] Traceback (most recent call last):
[2014-05-14 21:20:10,645: WARNING/MainProcess] File "manage.py", line 10, in 
[2014-05-14 21:20:10,645: WARNING/MainProcess] execute_from_command_line(sys.argv)
[2014-05-14 21:20:10,646: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/Django-1.6.1-py2.6.egg/django/core/management/__init.py", line 399, in execute_from_command_line
[2014-05-14 21:20:10,646: WARNING/MainProcess] utility.execute()
[2014-05-14 21:20:10,646: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/Django-1.6.1-py2.6.egg/django/core/management/init.py", line 392, in execute
[2014-05-14 21:20:10,647: WARNING/MainProcess] self.fetch_command(subcommand).run_from_argv(self.argv)
[2014-05-14 21:20:10,647: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/djcelery/management/base.py", line 77, in run_from_argv
[2014-05-14 21:20:10,647: WARNING/MainProcess] return super(CeleryCommand, self).run_from_argv(argv)
[2014-05-14 21:20:10,647: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/Django-1.6.1-py2.6.egg/django/core/management/base.py", line 242, in run_from_argv
[2014-05-14 21:20:10,648: WARNING/MainProcess] self.execute(*args, **options.dict)
[2014-05-14 21:20:10,648: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/djcelery/management/base.py", line 70, in execute
[2014-05-14 21:20:10,648: WARNING/MainProcess] super(CeleryCommand, self).execute(*args, **options)
[2014-05-14 21:20:10,649: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/Django-1.6.1-py2.6.egg/django/core/management/base.py", line 285, in execute
[2014-05-14 21:20:10,649: WARNING/MainProcess] output = self.handle(*args, **options)
[2014-05-14 21:20:10,649: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/djcelery/management/commands/celeryd.py", line 26, in handle
[2014-05-14 21:20:10,649: WARNING/MainProcess] worker.run(**options)
[2014-05-14 21:20:10,650: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/celery/bin/worker.py", line 212, in run
[2014-05-14 21:20:10,651: WARNING/MainProcess] state_db=self.node_format(state_db, hostname), **kwargs
[2014-05-14 21:20:10,651: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/celery/worker/init.py", line 211, in start
[2014-05-14 21:20:10,651: WARNING/MainProcess] self.stop()
[2014-05-14 21:20:10,652: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/celery/worker/init.py", line 252, in stop
[2014-05-14 21:20:10,652: WARNING/MainProcess] self.shutdown(warm=True)
[2014-05-14 21:20:10,652: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/celery/worker/__init_.py", line 266, in shutdown
[2014-05-14 21:20:10,652: WARNING/MainProcess] self.blueprint.stop(self, terminate=not warm)
[2014-05-14 21:20:10,653: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/celery/bootsteps.py", line 183, in stop
[2014-05-14 21:20:10,653: WARNING/MainProcess] self.on_stopped()
[2014-05-14 21:20:10,653: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/celery/worker/__init_.py", line 164, in on_stopped
[2014-05-14 21:20:10,653: WARNING/MainProcess] self.consumer.shutdown()
[2014-05-14 21:20:10,654: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/celery/worker/consumer.py", line 305, in shutdown
[2014-05-14 21:20:10,654: WARNING/MainProcess] self.blueprint.shutdown(self)
[2014-05-14 21:20:10,654: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/celery/worker/consumer.py", line 170, in shutdown
[2014-05-14 21:20:10,655: WARNING/MainProcess] self.send_all(parent, 'shutdown')
[2014-05-14 21:20:10,655: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/celery/bootsteps.py", line 155, in send_all
[2014-05-14 21:20:10,655: WARNING/MainProcess] fun(parent, *args)
[2014-05-14 21:20:10,655: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/celery/worker/consumer.py", line 531, in shutdown
[2014-05-14 21:20:10,655: WARNING/MainProcess] self._close(c)
[2014-05-14 21:20:10,656: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/celery/worker/consumer.py", line 525, in _close
[2014-05-14 21:20:10,659: WARNING/MainProcess] ignore_errors(c, dispatcher.connection.close)
[2014-05-14 21:20:10,659: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/kombu/common.py", line 273, in ignore_errors
[2014-05-14 21:20:10,660: WARNING/MainProcess] return fun(*args, **kwargs)
[2014-05-14 21:20:10,660: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/kombu/connection.py", line 334, in release
[2014-05-14 21:20:10,660: WARNING/MainProcess] self._close()
[2014-05-14 21:20:10,660: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/kombu/connection.py", line 302, in _close
[2014-05-14 21:20:10,661: WARNING/MainProcess] self._do_close_self()
[2014-05-14 21:20:10,661: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/kombu/connection.py", line 295, in _do_close_self
[2014-05-14 21:20:10,661: WARNING/MainProcess] self.transport.close_connection(self._connection)
[2014-05-14 21:20:10,661: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/kombu/transport/pyamqp.py", line 122, in close_connection
[2014-05-14 21:20:10,662: WARNING/MainProcess] connection.close()
[2014-05-14 21:20:10,663: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/amqp/connection.py", line 461, in close
2014-05-14 21:20:10,663: WARNING/MainProcess, # Connection.close_ok
[2014-05-14 21:20:10,663: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/amqp/abstract_channel.py", line 67, in wait
[2014-05-14 21:20:10,664: WARNING/MainProcess] self.channel_id, allowed_methods)
[2014-05-14 21:20:10,664: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/amqp/connection.py", line 270, in _wait_method
[2014-05-14 21:20:10,665: WARNING/MainProcess] self.wait()
[2014-05-14 21:20:10,665: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/amqp/abstract_channel.py", line 69, in wait
[2014-05-14 21:20:10,665: WARNING/MainProcess] return self.dispatch_method(method_sig, args, content)
[2014-05-14 21:20:10,665: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/amqp/abstract_channel.py", line 87, in dispatch_method
[2014-05-14 21:20:10,665: WARNING/MainProcess] return amqp_method(self, args)
[2014-05-14 21:20:10,666: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/amqp/connection.py", line 530, in _blocked
[2014-05-14 21:20:10,666: WARNING/MainProcess] reason = args.read_shortstr()
[2014-05-14 21:20:10,666: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/amqp/serialization.py", line 120, in read_shortstr
[2014-05-14 21:20:10,667: WARNING/MainProcess] return self.input.read(slen).decode('utf-8')
[2014-05-14 21:20:10,667: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/encodings/utf_8.py", line 16, in decode
[2014-05-14 21:20:10,667: WARNING/MainProcess] return codecs.utf_8_decode(input, errors, True)
[2014-05-14 21:20:10,668: WARNING/MainProcess] UnicodeDecodeError
[2014-05-14 21:20:10,668: WARNING/MainProcess] :
[2014-05-14 21:20:10,668: WARNING/MainProcess] 'utf8' codec can't decode byte 0xf5 in position 0: invalid start byte
[2014-05-14 21:20:10,668: DEBUG/MainProcess] removing tasks from inqueue until task handler finished

Answer 1

问题不是Unicode，而是阻止了与RabbitMQ的连接source。

原因是您可能正在使用apt存储库中过时的RabbitMQ（1.8.1）版本。您将在RabbitMQ日志中找到奇怪的错误（/ var / log / rabbitmq / *）source。

检查您的RabbitMQ版本：

sudo rabbitmqctl status

通过APT source

安装新版RabbitMQ

将以下行添加到/etc/apt/sources.list：

deb http://www.rabbitmq.com/debian/ testing main

（请注意，此行中的测试一词指的是我们发布的RabbitMQ的状态，而不是任何特定的Debian发行版。您可以将它与Debian稳定，测试或不稳定以及Ubuntu一起使用。我们描述了发布为“测试”以强调我们经常发布。）

（可选）要避免发出有关未签名包的警告，请使用apt-key（8）将我们的公钥添加到您的可信密钥列表中：

wget http://www.rabbitmq.com/rabbitmq-signing-key-public.asc
sudo apt-key add rabbitmq-signing-key-public.asc

apt-get update.

像往常一样安装包裹;例如，

sudo apt-get install rabbitmq-server

Answer 2

我遇到的另一种可能性是将我的队列配置从我的计算机上的文本文件复制粘贴到/etc/sysconfig/celeryd-worker-config中我服务器上的文件。一个unicode卷曲的报价悄悄进入并导致了这个问题。所以它与我的代码无关，而是我的配置。

UnicodeDecodeError：'utf8'编解码器无法解码 - 芹菜

2 个答案: