Question

我正在尝试使用基于此配置的命令celeryworker [1]运行芹菜[2]，但是当我启动程序时出现错误[3]。我的程序在medusa1-blank1中运行，rabbitmq-server正在hadoop-medusa-1中运行。您可以在[1]中看到$HOST_NAME变量是medusa1-blank1，celeryconfig.py包含正在运行rabbitmq-server的主机地址。

我查看了我的配置，我找不到任何错误。我希望日志可能更加冗长，以了解发生了什么，但我也不认为这样做是可能的。因为它看起来错误不在我的代码中，所以我完全无法理解正在发生的事情。尝试调试这个有什么帮助吗？

[1]我用芹菜运行的脚本

#!/bin/bash
set -xv

# This scripts runs celery in the server host

export C_FORCE_ROOT="true"
HOST_NAME=`hostname`

echo "------------------------"
echo "Initialize celery at $HOST_NAME"
echo "------------------------"
celery worker -n $HOST_NAME -E --loglevel=DEBUG --concurrency=20 -f ./logs/celerydebug.log --config=celeryconfig -Q $HOST_NAME 
# celery worker -n medusa1-blank1 -E --loglevel=DEBUG --concurrency=20 -f ./logs/celerydebug.log --config=celeryconfig -Q medusa1-blank1

[2]我使用的配置：

(medusa-env)xubuntu@medusa1-blank1:~/Programs/medusa-1.0$ cat celeryconfig.py
import os
import sys

# add hadoop python to the env, just for the running
sys.path.append(os.path.dirname(os.path.basename(__file__)))

# broker configuration
BROKER_URL = "amqp://celeryuser:celery@hadoop-medusa-1/celeryvhost"

CELERY_RESULT_BACKEND = "amqp"
CELERY_RESULT_PERSISTENT = True
TEST_RUNNER = 'celery.contrib.test_runner.run_tests'

# for debug
# CELERY_ALWAYS_EAGER = True

# module loaded
CELERY_IMPORTS = ("manager.mergedirs", "manager.system", "manager.utility", "manager.pingdaemon", "manager.hdfs")

[3]我有错误：

[2016-03-07 10:24:09,482: DEBUG/MainProcess] | Worker: Preparing bootsteps.
[2016-03-07 10:24:09,484: DEBUG/MainProcess] | Worker: Building graph...
[2016-03-07 10:24:09,484: DEBUG/MainProcess] | Worker: New boot order: {Timer, Hub, Queues (intra), Pool, Autoscaler, Autoreloader, StateDB, Beat, Consumer}
[2016-03-07 10:24:09,487: DEBUG/MainProcess] | Consumer: Preparing bootsteps.
[2016-03-07 10:24:09,487: DEBUG/MainProcess] | Consumer: Building graph...
[2016-03-07 10:24:09,491: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Agent, Events, Mingle, Tasks, Control, Heart, Gossip, event loop}
[2016-03-07 10:24:09,491: WARNING/MainProcess] /home/xubuntu/Programs/medusa-1.0/medusa-env/local/lib/python2.7/site-packages/celery/apps/worker.py:161: CDeprecationWarning: 
Starting from version 3.2 Celery will refuse to accept pickle by default.

The pickle serializer is a security concern as it may give attackers
the ability to execute any command.  It's important to secure
your broker from unauthorized access when using pickle, so we think
that enabling pickle should require a deliberate action and not be
the default choice.

If you depend on pickle then you should set a setting to disable this
warning and to be sure that everything will continue working
when you upgrade to Celery 3.2::

    CELERY_ACCEPT_CONTENT = ['pickle', 'json', 'msgpack', 'yaml']

You must only enable the serializers that you will actually use.


  warnings.warn(CDeprecationWarning(W_PICKLE_DEPRECATED))
[2016-03-07 10:24:09,493: ERROR/MainProcess] Unrecoverable error: AttributeError("'NoneType' object has no attribute 'rstrip'",)
Traceback (most recent call last):
  File "/home/xubuntu/Programs/medusa-1.0/medusa-env/local/lib/python2.7/site-packages/celery/worker/__init__.py", line 206, in start
    self.blueprint.start(self)
  File "/home/xubuntu/Programs/medusa-1.0/medusa-env/local/lib/python2.7/site-packages/celery/bootsteps.py", line 119, in start
    self.on_start()
  File "/home/xubuntu/Programs/medusa-1.0/medusa-env/local/lib/python2.7/site-packages/celery/apps/worker.py", line 169, in on_start
    string(self.colored.cyan(' \n', self.startup_info())),
  File "/home/xubuntu/Programs/medusa-1.0/medusa-env/local/lib/python2.7/site-packages/celery/apps/worker.py", line 230, in startup_info
    results=self.app.backend.as_uri(),
  File "/home/xubuntu/Programs/medusa-1.0/medusa-env/local/lib/python2.7/site-packages/celery/backends/base.py", line 117, in as_uri
    else maybe_sanitize_url(self.url).rstrip("/"))
AttributeError: 'NoneType' object has no attribute 'rstrip'

Answer 1

不知道您正在使用哪个版本，但发现此错误报告：

https://github.com/celery/celery/issues/3094

底线，暂时退回。

Answer 2

在我的最低配置文件中将是：

CELERY_IMPORTS = ...
AMPQ_USERNAME = os.getenv('AMQP_USERNAME', '...')
AMPQ_PASSWORD = os.getenv('AMQP_PASSWORD', '...')
AMQP_HOST = os.getenv('AMQP_HOST', '172.17.42.1')
AMQP_PORT = int(os.getenv('AMQP_PORT', '5672'))
DEFAULT_BROKER_URL = 'amqp://%s:%s@%s:%d'\
                     % (AMPQ_USERNAME, AMPQ_PASSWORD, AMQP_HOST, AMQP_PORT)

CELERY_RESULT_BACKEND = 'amqp://%s:%s@%s:%d'\
                     % (AMPQ_USERNAME, AMPQ_PASSWORD, AMQP_HOST, AMQP_PORT)

BROKER_API = DEFAULT_BROKER_URL

我启动芹菜时的AttributeError

2 个答案: