Question

您好我有python脚本，其中我创建了多个子进程来使用我的CPU的所有核心。由于python脚本需要时间，我决定使用芹菜排队。但现在当我使用芹菜运行我的python脚本时，我得到以下错误。

[2016-08-03 06:41:01,666: WARNING/Worker-4] Error: unable to start thread
[2016-08-03 06:41:01,666: WARNING/Worker-4] Traceback (most recent call last):
[2016-08-03 06:41:01,667: WARNING/Worker-4] File "/home/iaugmentor/django/iaugmentor/smile_detection/smile_eye_threading.py", line 310, in main
[2016-08-03 06:41:01,667: WARNING/Worker-4] job.start()
[2016-08-03 06:41:01,667: WARNING/Worker-4] File "/usr/lib/python2.7/multiprocessing/process.py", line 124, in start
[2016-08-03 06:41:01,681: WARNING/Worker-4] 'daemonic processes are not allowed to have children'
[2016-08-03 06:41:01,681: WARNING/Worker-4] AssertionError: daemonic processes are not allowed to have children

调用smile_detection函数并执行操作的python脚本。

    jobs = []
    batch_size = int(length/NUMBER_OF_CORES)

    # smile_detection("thread",images_list, 0, length)
    for i in range(NUMBER_OF_CORES):

        job = multiprocessing.Process(target=smile_detection, args=("Thread",images_list, i*batch_size, (i+1)*batch_size,))
        jobs.append(job)
        job.start()

    for j in jobs:
        j.join()

django views.py

class VideoDetails(APIView):

def get(self, request, *args, **kwargs):
    return render(request, "smile_detection/index1.html")

def options(self, request, *args, **kwargs):
    return Response()

def post(self, request, *args, **kwargs):

    analysevideo.delay(request.data)
    # smile_eye_threading.main(request.data)
    return Response(status=HTTP_201_CREATED)

tasks.py

from __future__ import absolute_import

from celery.decorators import task
from . import smile_eye_threading

@task(name="analyse video")
def analysevideo(data):
smile_eye_threading.main(data)

celery.py

 from __future__ import absolute_import
 import os
 import django
 from celery import Celery
 from django.conf import settings

 # set the default Django settings module for the 'celery' program.
 os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'iaugmentor.settings')
 django.setup()                                     
 app = Celery('iaugmentor')

 # Using a string here means the worker will not have to
 # pickle the object when using Windows.
 app.config_from_object('django.conf:settings')
 app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)


 @app.task(bind=True)
 def debug_task(self):
 print('Request: {0!r}'.format(self.request))

setting.py我添加了以下行

# CELERY STUFF
BROKER_URL = 'redis://localhost:6379'
CELERY_RESULT_BACKEND = 'redis://localhost:6379'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TIMEZONE = 'Africa/Nairobi'

现在该怎么办。我只是希望我的工作排队等候。我应该为此写一份cron工作吗？我已转到此链接：https://github.com/celery/celery/issues/1709 但我无法理解它。

无法使用celery任务启动任务中的线程

0 个答案: