尝试在Celery Beat中添加RecursionError / KeyError尝试添加定期任务

时间：2018-08-28 17:21:50

标签： python django celery celerybeat kombu

我尝试在Django上使用Celery beat设置一些定期任务。 我正在使用下一个设置：

python3.6
django==1.11.10
celery==4.2.1
kombu==4.2.1

我的日程表工作程序通过docker-compose在Docker容器中运行，并从下一条命令开始：

celery -A proj beat --loglevel=debug -s /src/celerybeat-schedule

没有添加定期任务，一切正常，我在控制台中获得了以下代码行：

[2018-08-28 17:06:05,261: DEBUG/MainProcess] Setting default socket timeout to 30
[2018-08-28 17:06:05,267: INFO/MainProcess] beat: Starting...
[2018-08-28 17:06:05,298: DEBUG/MainProcess] Current schedule:

[2018-08-28 17:06:05,298: DEBUG/MainProcess] beat: Ticking with max interval->5.00 minutes
[2018-08-28 17:06:05,299: DEBUG/MainProcess] beat: Waking up in 5.00 minutes.

但是当我尝试添加定期任务时（紧接Celery设置之后）：

@app.on_after_configure.connect
def setup_periodic_tasks(sender):
    sender.add_periodic_task(10.0, test.s('hello'), name='add every 10')


@app.task()
def test(arg):
    print(arg)

我在这行中得到KeyError：

[2018-08-28 17:11:56,666: WARNING/MainProcess] Traceback (most recent call last):
[2018-08-28 17:11:56,666: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/kombu/utils/objects.py", line 42, in __get__
[2018-08-28 17:11:56,666: WARNING/MainProcess] return obj.__dict__[self.__name__]
[2018-08-28 17:11:56,666: WARNING/MainProcess] KeyError
[2018-08-28 17:11:56,666: WARNING/MainProcess] :
[2018-08-28 17:11:56,667: WARNING/MainProcess] 'tz'

然后是'timezone'和'data'的相同错误

然后是很多（RecursionError）：

[2018-08-28 17:11:57,050: WARNING/MainProcess] File "/usr/local/lib/python3.6/collections/__init__.py", line 987, in __getitem__
[2018-08-28 17:11:57,050: WARNING/MainProcess] if key in self.data:
[2018-08-28 17:11:57,051: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/utils/collections.py", line 126, in __getattr__
[2018-08-28 17:11:57,051: WARNING/MainProcess] return self[k]
[2018-08-28 17:11:57,051: WARNING/MainProcess] File "/usr/local/lib/python3.6/collections/__init__.py", line 987, in __getitem__
[2018-08-28 17:11:57,052: WARNING/MainProcess] if key in self.data:
[2018-08-28 17:11:57,052: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/utils/collections.py", line 126, in __getattr__
[2018-08-28 17:11:57,052: WARNING/MainProcess] return self[k]

最后的字符串是：

[2018-08-28 17:11:57,115: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/kombu/utils/imports.py", line 56, in symbol_by_name
[2018-08-28 17:11:57,116: WARNING/MainProcess] module = imp(module_name, package=package, **kwargs)
[2018-08-28 17:11:57,116: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/loaders/base.py", line 98, in import_from_cwd
[2018-08-28 17:11:57,117: WARNING/MainProcess] package=package,
[2018-08-28 17:11:57,117: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/utils/imports.py", line 104, in import_from_cwd
[2018-08-28 17:11:57,117: WARNING/MainProcess] return imp(module, package=package)
[2018-08-28 17:11:57,118: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/loaders/base.py", line 92, in import_module
[2018-08-28 17:11:57,118: WARNING/MainProcess] return importlib.import_module(module, package=package)
[2018-08-28 17:11:57,118: WARNING/MainProcess] File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module
[2018-08-28 17:11:57,119: WARNING/MainProcess] return _bootstrap._gcd_import(name[level:], package, level)
[2018-08-28 17:11:57,119: WARNING/MainProcess] File "<frozen importlib._bootstrap>", line 994, in _gcd_import
[2018-08-28 17:11:57,121: WARNING/MainProcess] File "<frozen importlib._bootstrap>", line 968, in _find_and_load
[2018-08-28 17:11:57,122: WARNING/MainProcess] RecursionError
[2018-08-28 17:11:57,123: WARNING/MainProcess] :
[2018-08-28 17:11:57,123: WARNING/MainProcess] maximum recursion depth exceeded

大约一个星期无法找到解决方案，所以我在这里。我已经准备好为您提供尽可能多的设置。

谢谢：）

非常大的更新，对不起，购买也许很重要。这是KeyErrors和递归开始之间发生的事情：

[2018-08-28 17:46:18,467: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/kombu/utils/objects.py", line 42, in __get__
[2018-08-28 17:46:18,468: WARNING/MainProcess] return obj.__dict__[self.__name__]
[2018-08-28 17:46:18,468: WARNING/MainProcess] KeyError
[2018-08-28 17:46:18,468: WARNING/MainProcess] :
[2018-08-28 17:46:18,468: WARNING/MainProcess] 'data'
[2018-08-28 17:46:18,469: WARNING/MainProcess] During handling of the above exception, another exception occurred:
[2018-08-28 17:46:18,469: WARNING/MainProcess] Traceback (most recent call last):
[2018-08-28 17:46:18,469: WARNING/MainProcess] File "/usr/local/bin/celery", line 11, in <module>
[2018-08-28 17:46:18,470: WARNING/MainProcess] sys.exit(main())
[2018-08-28 17:46:18,470: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/__main__.py", line 16, in main
[2018-08-28 17:46:18,470: WARNING/MainProcess] _main()
[2018-08-28 17:46:18,470: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 322, in main
[2018-08-28 17:46:18,471: WARNING/MainProcess] cmd.execute_from_commandline(argv)
[2018-08-28 17:46:18,471: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 496, in execute_from_commandline
[2018-08-28 17:46:18,473: WARNING/MainProcess] super(CeleryCommand, self).execute_from_commandline(argv)))
[2018-08-28 17:46:18,473: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 275, in execute_from_commandline
[2018-08-28 17:46:18,474: WARNING/MainProcess] return self.handle_argv(self.prog_name, argv[1:])
[2018-08-28 17:46:18,475: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 488, in handle_argv
[2018-08-28 17:46:18,476: WARNING/MainProcess] return self.execute(command, argv)
[2018-08-28 17:46:18,476: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 420, in execute
[2018-08-28 17:46:18,477: WARNING/MainProcess] ).run_from_argv(self.prog_name, argv[1:], command=argv[0])
[2018-08-28 17:46:18,477: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 279, in run_from_argv
[2018-08-28 17:46:18,477: WARNING/MainProcess] sys.argv if argv is None else argv, command)
[2018-08-28 17:46:18,478: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 363, in handle_argv
[2018-08-28 17:46:18,478: WARNING/MainProcess] return self(*args, **options)
[2018-08-28 17:46:18,479: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 238, in __call__
[2018-08-28 17:46:18,479: WARNING/MainProcess] ret = self.run(*args, **kwargs)
[2018-08-28 17:46:18,480: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/bin/beat.py", line 109, in run
[2018-08-28 17:46:18,480: WARNING/MainProcess] return beat().run()
[2018-08-28 17:46:18,481: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/apps/beat.py", line 81, in run
[2018-08-28 17:46:18,481: WARNING/MainProcess] self.start_scheduler()
[2018-08-28 17:46:18,482: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/apps/beat.py", line 109, in start_scheduler
[2018-08-28 17:46:18,482: WARNING/MainProcess] service.start()
[2018-08-28 17:46:18,483: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/beat.py", line 588, in start
[2018-08-28 17:46:18,483: WARNING/MainProcess] interval = self.scheduler.tick()
[2018-08-28 17:46:18,484: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/beat.py", line 293, in tick
[2018-08-28 17:46:18,484: WARNING/MainProcess] self.populate_heap()
[2018-08-28 17:46:18,484: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/beat.py", line 267, in populate_heap
[2018-08-28 17:46:18,485: WARNING/MainProcess] is_due, next_call_delay = entry.is_due()
[2018-08-28 17:46:18,485: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/beat.py", line 132, in is_due
[2018-08-28 17:46:18,485: WARNING/MainProcess] return self.schedule.is_due(self.last_run_at)
[2018-08-28 17:46:18,486: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/schedules.py", line 163, in is_due
[2018-08-28 17:46:18,486: WARNING/MainProcess] last_run_at = self.maybe_make_aware(last_run_at)
[2018-08-28 17:46:18,487: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/schedules.py", line 78, in maybe_make_aware
[2018-08-28 17:46:18,487: WARNING/MainProcess] return maybe_make_aware(dt, self.tz)
[2018-08-28 17:46:18,488: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/kombu/utils/objects.py", line 44, in __get__
[2018-08-28 17:46:18,488: WARNING/MainProcess] value = obj.__dict__[self.__name__] = self.__get(obj)
[2018-08-28 17:46:18,488: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/schedules.py", line 90, in tz
[2018-08-28 17:46:18,489: WARNING/MainProcess] return self.app.timezone
[2018-08-28 17:46:18,489: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/kombu/utils/objects.py", line 44, in __get__
[2018-08-28 17:46:18,489: WARNING/MainProcess] value = obj.__dict__[self.__name__] = self.__get(obj)
[2018-08-28 17:46:18,490: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/app/base.py", line 1260, in timezone
[2018-08-28 17:46:18,491: WARNING/MainProcess] if not conf.timezone:
[2018-08-28 17:46:18,491: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/app/utils.py", line 129, in timezone
[2018-08-28 17:46:18,491: WARNING/MainProcess] return self.first('timezone', 'time_zone')
[2018-08-28 17:46:18,492: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/utils/collections.py", line 447, in first
[2018-08-28 17:46:18,492: WARNING/MainProcess] return first(None, (self.get(key) for key in keys))
[2018-08-28 17:46:18,493: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/utils/functional.py", line 84, in first
[2018-08-28 17:46:18,493: WARNING/MainProcess] None,
[2018-08-28 17:46:18,493: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/utils/functional.py", line 82, in <genexpr>
[2018-08-28 17:46:18,494: WARNING/MainProcess] (v for v in evaluate_promises(it) if (
[2018-08-28 17:46:18,494: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/utils/functional.py", line 69, in evaluate_promises
[2018-08-28 17:46:18,494: WARNING/MainProcess] for value in it:
[2018-08-28 17:46:18,495: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/utils/collections.py", line 447, in <genexpr>
[2018-08-28 17:46:18,495: WARNING/MainProcess] return first(None, (self.get(key) for key in keys))
[2018-08-28 17:46:18,496: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/utils/collections.py", line 452, in get
[2018-08-28 17:46:18,496: WARNING/MainProcess] return self[key]
[2018-08-28 17:46:18,497: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/utils/collections.py", line 429, in __getitem__
[2018-08-28 17:46:18,497: WARNING/MainProcess] return getitem(k)
[2018-08-28 17:46:18,498: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/utils/collections.py", line 278, in __getitem__
[2018-08-28 17:46:18,498: WARNING/MainProcess] return mapping[_key]
[2018-08-28 17:46:18,499: WARNING/MainProcess] File "/usr/local/lib/python3.6/collections/__init__.py", line 987, in __getitem__
[2018-08-28 17:46:18,499: WARNING/MainProcess] if key in self.data:
[2018-08-28 17:46:18,500: WARNING/MainProcess] File "/usr/local/lib/python3.6/site-packages/celery/utils/collections.py", line 126, in __getattr__
[2018-08-28 17:46:18,500: WARNING/MainProcess] return self[k]

我还尝试为celery_app设置tz /时区：

app.conf.timezone = pytz.timezone('UTC')
app.conf.tz = pytz.timezone('UTC')
app.timezone = pytz.timezone('UTC')
app.tz = pytz.timezone('UTC')

我在Django中也有TZ设置...

我也发现了日志的开始，但这实际上是我的最后一部分：（

[2018-08-28 18:01:56,536: DEBUG/MainProcess] Setting default socket timeout to 30
[2018-08-28 18:01:56,537: INFO/MainProcess] beat: Starting...
[2018-08-28 18:01:56,558: DEBUG/MainProcess] Current schedule:
<ScheduleEntry: add every 10 somewhere_api.celery.test('hello') <freq: 10.00 seconds>
[2018-08-28 18:01:56,558: DEBUG/MainProcess] beat: Ticking with max interval->5.00 minutes
[2018-08-28 18:01:56,572: CRITICAL/MainProcess] beat raised exception <class 'RecursionError'>: RecursionError('maximum recursion depth exceeded',)
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/kombu/utils/objects.py", line 42, in __get__
    return obj.__dict__[self.__name__]
KeyError: 'tz'

2 个答案:

答案 0 :(得分：0)

我发现了错误，解决方法是要小心。我的Django旧版应用程序中的一个突然出现了另一个Celery实例，该实例试图从其他设置开始进行操作，并且破坏了一切。

答案 1 :(得分：0)

我刚刚遇到了类似的问题，但我只使用 Celery 和 RabbitMQ，最后我删除了 celerybeat-schedule.db 并重新运行应用程序，然后就可以了。

相关问题
最新问题