Question

我正在将一个带有Celery的Django应用程序部署到Heroku并且遇到连接错误。我的AMQP提供商声称他能够访问他的资源，并且我有足够的连接。我认为我的设置是在我的特定工作进程中找到芹菜应用程序，但未能在shell中获取正确的设置。有没有办法告诉调用任务的连接URL是什么？我的celery_app检查下面应该正确处理事情吗？

问题调用：

$ heroku run bash
~$ python <app>/manage.py shell
>>> from <app>.management.tasks.tasks import <task>
>>> t = <task>()
>>> dt = '20140101'
>>> t.delay(dt=dt)

堆栈跟踪：

Traceback (most recent call last):
  File "<input>", line 4, in <module>
  File "/app/.heroku/python/lib/python2.7/site-packages/celery/app/task.py", line 453, in delay
    return self.apply_async(args, kwargs)
  File "/app/.heroku/python/lib/python2.7/site-packages/celery/app/task.py", line 555, in apply_async
    **dict(self._get_exec_options(), **options)
  File "/app/.heroku/python/lib/python2.7/site-packages/celery/app/base.py", line 353, in send_task
    reply_to=reply_to or self.oid, **options
  File "/app/.heroku/python/lib/python2.7/site-packages/celery/app/amqp.py", line 305, in publish_task
    **kwargs
  File "/app/.heroku/python/lib/python2.7/site-packages/kombu/messaging.py", line 168, in publish
    routing_key, mandatory, immediate, exchange, declare)
  File "/app/.heroku/python/lib/python2.7/site-packages/kombu/connection.py", line 457, in _ensured
    interval_max)
  File "/app/.heroku/python/lib/python2.7/site-packages/kombu/connection.py", line 369, in ensure_connection
    interval_start, interval_step, interval_max, callback)
  File "/app/.heroku/python/lib/python2.7/site-packages/kombu/utils/__init__.py", line 243, in retry_over_time
    return fun(*args, **kwargs)
  File "/app/.heroku/python/lib/python2.7/site-packages/kombu/connection.py", line 237, in connect
    return self.connection
  File "/app/.heroku/python/lib/python2.7/site-packages/kombu/connection.py", line 741, in connection
    self._connection = self._establish_connection()
  File "/app/.heroku/python/lib/python2.7/site-packages/kombu/connection.py", line 696, in _establish_connection
    conn = self.transport.establish_connection()
  File "/app/.heroku/python/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 112, in establish_connection
    conn = self.Connection(**opts)
  File "/app/.heroku/python/lib/python2.7/site-packages/amqp/connection.py", line 165, in __init__
    self.transport = create_transport(host, connect_timeout, ssl)
  File "/app/.heroku/python/lib/python2.7/site-packages/amqp/transport.py", line 294, in create_transport
    return TCPTransport(host, connect_timeout)
  File "/app/.heroku/python/lib/python2.7/site-packages/amqp/transport.py", line 95, in __init__
    raise socket.error(last_err)
error: [Errno 111] Connection refused

在shell中检查配置：

~$ python <app>/manage.py shell
>>> from celery_app import Config
>>> Config.BROKER_URL
'<correct amqp resource>'

Env vars：

CELERYD_CONCURRENCY=1
CELERY_IGNORE_RESULT=True
CELERYD_TASK_TIME_LIMIT=60
CLOUDAMQP_URL=<amqp url>
RESULT_EXPIRY_RATE=600
BROKER_CONNECTION_TIMEOUT=10
PWD=/app
DJANGO_SETTINGS_MODULE=settings.production
DJANGO_PROJECT_DIR=/app/<app>
BROKER_POOL_LIMIT=1
HOME=/app
PYTHONPATH=/app:/app/<app>:/app/<app>/<app>

Procfile调用（有效）

web: gunicorn <app>.<app>.wsgi -w 1 --log-file -
worker: celery worker --app=<app>.<app> -E -Q <app>,celery --loglevel=INFO  -c 1  --workdir=<app>

我的芹菜应用程序：

from __future__ import absolute_import
from os import getenv
from kombu import Exchange, Queue
from django.conf import settings
from celery import Celery


app = Celery('<app>')

class Config(object):

    # List of modules to import when celery starts.
    CELERY_IMPORTS = ("<imports>",)


    BROKER_CONNECTION_RETRY = True
    API_RATE_LIMIT = getenv('API_RATE_LIMIT')
    BROKER_POOL_LIMIT = int(getenv('BROKER_POOL_LIMIT', 1))
    BROKER_URL = getenv('CLOUDAMQP_URL')
    BROKER_CONNECTION_TIMEOUT = int(getenv('BROKER_CONNECTION_TIMEOUT'))
    CELERYD_CONCURRENCY = int(getenv('CELERYD_CONCURRENCY'))


app.config_from_object(Config)

if __name__ == '__main__':
    app.start()

在同一目录中的 init .py中导入应用：

from __future__ import absolute_import

# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .celery_app import app as celery_app  # nolint

Answer 1

核心问题是django，celery和gunicorn与DJANGO_SETTINGS_MODULE的不同预期格式。我通过将DJANGO_SETTINGS_MODULE=settings.production更改为DJANGO_SETTINGS_MODULE=<app>.settings.production来修复此问题，这修复了shell代理连接但破坏了我的Web和工作进程。一个有效的Procfile规范是

web: cd <app> && gunicorn <app>.wsgi -w 1 --log-file -
worker: celery worker --app=<app> -E -Q <app>,celery --loglevel=INFO  -c 1  --workdir=<app>

网络之前的CD很傻，但有效。

Answer 2

当我将Django项目部署到Tornado Server时，我遇到了这个异常。这是部署代码：

import os
import tornado.httpserver
import tornado.ioloop
import tornado.wsgi
from django.core.wsgi import get_wsgi_application

# add this line when use celery.
# import app

def main():
    os.environ['DJANGO_SETTINGS_MODULE'] = 'myproject.settings' # path to your settings module
    application = get_wsgi_application()
    container = tornado.wsgi.WSGIContainer(application)
    http_server = tornado.httpserver.HTTPServer(container)
    http_server.listen(8888)
    tornado.ioloop.IOLoop.instance().start()

if __name__ == "__main__":
    main()

当我添加导入应用（包含celery app init文件的应用）时，一切正常。

芹菜连接错误

2 个答案: