Question

我的项目目录结构如下：

rss_reader  
 - reader  
    - __init__.py   
    - models.py  
    - views.py  
    - ...  
 - rss_reader  
    - __init__.py   
    - settings.py  
    - urls.py  
    - wsgi.py  
 - rss_contents  
    - __init__.py   
    - celery.py  
    - tasks.py  
    - config.py  
 - ...

tasks.py

from __future__ import absolute_import
from rss_contents.celery import app

@app.task
def processArticle():
    print "100"

celery.py

from __future__ import absolute_import
from celery import Celery

app = Celery('rss_contents', include=['rss_contents.tasks'])
app.config_from_object('rss_contents.config')

config.py

from __future__ import absolute_import

CELERY_RESULT_BACKEND = 'redis://127.0.0.1:6379/5'
BROKER_URL = 'redis://127.0.0.1:6379/6'

读者＆gt; __init __。PY

from rss_contents import tasks
tasks.processArticle.delay()

我的步骤如下：

celery -A rss_contents worker -l info

控制台显示：

[2017-01-11 10:34:33,829: INFO/MainProcess] Connected to redis://127.0.0.1:6379/6  
[2017-01-11 10:34:33,855: INFO/MainProcess] mingle: searching for neighbors  
[2017-01-11 10:34:34,861: INFO/MainProcess] mingle: all alone  
[2017-01-11 10:34:34,892: WARNING/MainProcess] celery@DESKTOP-6KAT7MF ready.

运行服务器，在http://127.0.0.1:8000/

但芹菜控制台显示该任务已运行两次：

[2017-01-11 10:41:20,910: INFO/MainProcess] Received task: rss_contents.tasks.processArticle[aa355c77-4ee8-4208-9e8c-915b110c7bbd]  
[2017-01-11 10:41:20,911: WARNING/Worker-1] 100
[2017-01-11 10:41:20,917: INFO/MainProcess] Task rss_contents.tasks.processArticle[aa355c77-4ee8-4208-9e8c-915b110c7bbd] succeeded in 0.00600004196167s: None  

[2017-01-11 10:41:22,430: INFO/MainProcess] Received task: rss_contents.tasks.processArticle[d92a151c-f0f9-4e8f-921a-fff2c1eb64c6]  
[2017-01-11 10:41:22,431: WARNING/Worker-1] 100   
[2017-01-11 10:41:22,447: INFO/MainProcess] Task rss_contents.tasks.processArticle[d92a151c-f0f9-4e8f-921a-fff2c1eb64c6] succeeded in 0.0160000324249s: None

为什么会跑两次？我怎么能这样做？

提前致谢。

Answer 1

那是因为您已在__init__.py文件中定义了任务调用。

我已经开始使用您的代码进行测试，并在__init__.py内插入一个pdb中断

from celery_proj import tasks
import pdb; pdb.set_trace()  # <-- make a break point here
print __file__
tasks.processArticle.delay()

并获得以下兴趣发现：

当您运行python manage.py runserver时，断点将使用以下堆栈跟踪触发一次：

(Pdb) where
  /Users/enix/Source/python/stackoverflow/sotest/manage.py(10)<module>()
-> execute_from_command_line(sys.argv)
  /Users/enix/Source/python/bae/lib/python2.7/site-packages/django/core/management/__init__.py(353)execute_from_command_line()
-> utility.execute()
  /Users/enix/Source/python/bae/lib/python2.7/site-packages/django/core/management/__init__.py(316)execute()
-> autoreload.check_errors(django.setup)()
  /Users/enix/Source/python/bae/lib/python2.7/site-packages/django/utils/autoreload.py(226)wrapper()
-> fn(*args, **kwargs)
  /Users/enix/Source/python/bae/lib/python2.7/site-packages/django/__init__.py(18)setup()
-> apps.populate(settings.INSTALLED_APPS)
  /Users/enix/Source/python/bae/lib/python2.7/site-packages/django/apps/registry.py(85)populate()
-> app_config = AppConfig.create(entry)
  /Users/enix/Source/python/bae/lib/python2.7/site-packages/django/apps/config.py(90)create()
-> module = import_module(entry)
  /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/importlib/__init__.py(37)import_module()
-> __import__(name)
> /Users/enix/Source/python/stackoverflow/sotest/app/__init__.py(4)<module>()
-> print __file__

当我键入c继续时，断点将再次触发，并具有相同的堆栈跟踪。

所以你的任务被调用了两次。但我不太清楚为什么django会两次导入应用程序包。

(bae)bogon:sotest enix$ python manage.py runserver
/Users/enix/Source/python/stackoverflow/sotest/app/__init__.py  # <-- print __file__
/Users/enix/Source/python/stackoverflow/sotest/app/__init__.pyc # <-- print __file__
Performing system checks...
System check identified no issues (0 silenced).
January 10, 2017 - 21:44:12
Django version 1.9.12, using settings 'sotest.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

也许你应该把你的任务电话转移到其他地方再试一次。

Answer 2

@enix把它钉在头上。因为reader是一个应用程序，所以当django初始化应用程序注册表时会调用它一次。但是，由于这种调用是动态完成的，因此读者应用程序可能会第二次初始化，因为某处读者包的全局使用。在任何情况下，解决此问题的最简单方法是将您的调用移动到一个类，并从任何python文件中的单例调用它。

class AppInitializer(object):
    initialized = false

    @classmethod 
    def initialize(cls):
        if not cls.initialized:
            cls.initialized = True
            from rss_contents import tasks
            tasks.processArticle.delay()

AppInitializer.initialize()

如果您的appengine是多线程的，那么您需要在初始化时添加方法锁。

Django Celery任务运行两次

我的步骤如下：

2 个答案: