Question

尝试使用mongodb代理作为后端获取Celery设置（遵循基本教程）。遵循官方文档中列出的配置指南，我的celeryconfig.py设置如下：

CELERY_RESULT_BACKEND = "mongodb"

BROKER_BACKEND = "mongodb"
BROKER_URL = "mongodb://user:pass@subdomain.mongolab.com:123456/testdb"

CELERY_MONGODB_BACKEND_SETTINGS = {
    "host":"subdomain.mongolab.com",
    "port":123456,
    "database":"testdb",
    "taskmeta_collection":"taskmeta",
    "user":"user",
    "pass":"pass",
}
CELERY_IMPORTS = ("tasks",)

使用--loglevel=INFO运行celeryd会返回以下异常，从pymongo开始，但在kombu和芹菜中冒泡。

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/celery/worker/__init__.py", line 230, in start
    component.start()
  File "/usr/local/lib/python2.7/dist-packages/celery/worker/consumer.py", line 338, in start
    self.reset_connection()
  File "/usr/local/lib/python2.7/dist-packages/celery/worker/consumer.py", line 596, in reset_connection
    on_decode_error=self.on_decode_error)
  File "/usr/local/lib/python2.7/dist-packages/celery/app/amqp.py", line 335, in get_task_consumer
    **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/kombu/compat.py", line 187, in __init__
    super(ConsumerSet, self).__init__(self.backend, queues, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/kombu/messaging.py", line 285, in __init__
    self.declare()
  File "/usr/local/lib/python2.7/dist-packages/kombu/messaging.py", line 295, in declare
    queue.declare()
  File "/usr/local/lib/python2.7/dist-packages/kombu/entity.py", line 388, in declare
    self.queue_declare(nowait, passive=False)
  File "/usr/local/lib/python2.7/dist-packages/kombu/entity.py", line 408, in queue_declare
    nowait=nowait)
  File "/usr/local/lib/python2.7/dist-packages/kombu/transport/virtual/__init__.py", line 380, in queue_declare
    return queue, self._size(queue), 0
  File "/usr/local/lib/python2.7/dist-packages/kombu/transport/mongodb.py", line 74, in _size
    return self.client.messages.find({"queue": queue}).count()
  File "/usr/local/lib/python2.7/dist-packages/kombu/transport/mongodb.py", line 171, in client
    self._client = self._open()
  File "/usr/local/lib/python2.7/dist-packages/kombu/transport/mongodb.py", line 97, in _open
    mongoconn = Connection(host=conninfo.hostname, port=conninfo.port)
  File "/usr/local/lib/python2.7/dist-packages/pymongo/connection.py", line 325, in __init__
    nodes.update(uri_parser.split_hosts(entity, port))
  File "/usr/local/lib/python2.7/dist-packages/pymongo/uri_parser.py", line 198, in split_hosts
    nodes.append(parse_host(entity, default_port))
  File "/usr/local/lib/python2.7/dist-packages/pymongo/uri_parser.py", line 127, in parse_host
    raise ConfigurationError("Reserved characters such as ':' must be "
ConfigurationError: Reserved characters such as ':' must be escaped according RFC 2396. An IPv6 address literal must be enclosed in '[' and ']' according to RFC 2732.

关于Celery处理mongouri的方式有些不正确，因为pymongo中的uri解析器会抛出此错误。我已经尝试转义uri字符串中的:个字符，但所有这一切都会将传输重置为带有错位连接字符串的默认AMQP。

amqp://guest@localhost:5672/mongodb\http://user\:password@subdomain.mongolab.com\:29217/testdb

显然这是不对的。

我尝试使用r将配置中的uri作为原始字符串输入，但没有任何变化。

我知道自从2.4（我使用的是2.5.1，pymongo 2.1.1）以来，Celery一直支持这种连接配置，官方文档都将它作为连接mongodb代理的首选方法。< / p>

这可能是一个错误，也许与最新的pymongo构建不兼容？如果这种方法不起作用，那么如何将任务队列附加到副本集，因为我认为这些必须使用?replicaSet参数在mongouri中传递。

我应该注意，我宁愿不转而使用RabbitMQ代理，因为Mongo已经在相关应用程序的堆栈中，并且使用已经存在的内容似乎更直观。如果有一个具体的原因，为什么Mongo会为此目的不那么有效（每天的任务量相对较小）我很想知道！提前谢谢。

Answer 1

我认为这是一个错误。 Celery将主机名而不是server_uri传递给kombu，从而导致此问题。在跟踪代码之后，我发现以下内容可以在修复之前绕过该错误。

CELERY_RESULT_BACKEND = 'mongodb'
BROKER_HOST = "subdomain.mongolab.com"
BROKER_PORT = 123456
BROKER_TRANSPORT = 'mongodb'
BROKER_VHOST = 'testdb'
CELERY_IMPORTS = ('tasks',)
CELERY_MONGODB_BACKEND_SETTINGS = { 
       'host': 'subdomain.mongolab.com',
       'port': 123456,
       'database': 'testdb',
       'user': user,
       'password': password,
       'taskmeta_collection': 'teskmeta'
       }

重复配置。

Answer 2

如果从CELERY_MONGODB_BACKEND_SETTINGS dict中删除“user”，“pass”，“port”和“database”，它会有所帮助，并执行：

BROKER_URL = "mongodb://user:pass@subdomain.mongolab.com:123456/testdb"

CELERY_MONGODB_BACKEND_SETTINGS = {
    "host":BROKER_URL,
    "taskmeta_collection":"taskmeta",
}

芹菜与MongoDB经纪人

2 个答案: