如何使用多租户(django-tenant-schemas)Django应用程序设置django-celery-beat。

时间:2018-05-04 18:38:15

标签: django celery

我正试着和芹菜一起吃芹菜。 Celery与我的多租户Django应用程序(使用django-tenant-schemas)在tenant_schemas_celery的帮助下工作正常。但是由于所有预定的任务/周期性任务相关表都是特定于租户的,因此我无法运行芹菜节拍,因此无法启动芹菜节拍。

错误追踪:

celery@Amits-iMac.local v4.0.2 (latentcall)

Darwin-17.5.0-x86_64-i386-64bit 2018-05-04 18:20:22

[config]
.> app:         __main__:0x102ef9f28
.> transport:   amqp://guest:**@localhost:5672//
.> results:
.> concurrency: 4 (prefork)
.> task events: ON

[queues]
.> celery           exchange=celery(direct) key=celery


[tasks]
  . dataflow.tasks.launch_dataflow
  . datasession.tasks.launch_copy

[2018-05-04 18:20:23,035: INFO/Beat] beat: Starting...
[2018-05-04 18:20:23,166: ERROR/Beat] Process Beat
Traceback (most recent call last):
  File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/kombu/utils/objects.py", line 42, in __get__
    return obj.__dict__[self.__name__]
KeyError: 'scheduler'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
psycopg2.ProgrammingError: relation "django_celery_beat_periodictask" does not exist
LINE 1: ...ango_celery_beat_periodictask"."description" FROM "django_ce...
                                                             ^


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/billiard/process.py", line 306, in _bootstrap
    self.run()
  File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/celery/beat.py", line 613, in run
    self.service.start(embedded_process=True)
  File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/celery/beat.py", line 528, in start
    humanize_seconds(self.scheduler.max_interval))
  File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/kombu/utils/objects.py", line 44, in __get__
    value = obj.__dict__[self.__name__] = self.__get(obj)
  File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/celery/beat.py", line 572, in scheduler
    return self.get_scheduler()
  File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/celery/beat.py", line 567, in get_scheduler
    lazy=lazy,
  File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django_celery_beat/schedulers.py", line 181, in __init__
    Scheduler.__init__(self, *args, **kwargs)
  File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/celery/beat.py", line 204, in __init__
    self.setup_schedule()
  File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django_celery_beat/schedulers.py", line 189, in setup_schedule
    self.install_default_entries(self.schedule)
  File "/Users/amit/thoughtanalytik/koolanch/src/koolanch/beat_schedulers.py", line 27, in schedule
    self._schedule = self.all_as_schedule()
  File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django_celery_beat/schedulers.py", line 195, in all_as_schedule
    for model in self.Model.objects.enabled():
  File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django/db/models/query.py", line 250, in __iter__
    self._fetch_all()
  File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django/db/models/query.py", line 1118, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django/db/models/query.py", line 53, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch)
  File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 894, in execute_sql
    raise original_exception
  File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 884, in execute_sql
    cursor.execute(sql, params)
  File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django/db/backends/utils.py", line 80, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "django_celery_beat_periodictask" does not exist
LINE 1: ...ango_celery_beat_periodictask"."description" FROM "django_ce...
                                                             ^

[2018-05-04 18:20:23,219: WARNING/Beat] Process Beat:
[2018-05-04 18:20:23,220: WARNING/Beat] Traceback (most recent call last):
[2018-05-04 18:20:23,220: WARNING/Beat] File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/kombu/utils/objects.py", line 42, in __get__
    return obj.__dict__[self.__name__]
[2018-05-04 18:20:23,220: WARNING/Beat] KeyError: 'scheduler'
[2018-05-04 18:20:23,220: WARNING/Beat] During handling of the above exception, another exception occurred:
[2018-05-04 18:20:23,220: WARNING/Beat] Traceback (most recent call last):
[2018-05-04 18:20:23,221: WARNING/Beat] File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
[2018-05-04 18:20:23,221: WARNING/Beat] psycopg2.ProgrammingError: relation "django_celery_beat_periodictask" does not exist
LINE 1: ...ango_celery_beat_periodictask"."description" FROM "django_ce...
                                                             ^
[2018-05-04 18:20:23,221: WARNING/Beat] The above exception was the direct cause of the following exception:
[2018-05-04 18:20:23,221: WARNING/Beat] Traceback (most recent call last):
[2018-05-04 18:20:23,221: WARNING/Beat] File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/billiard/process.py", line 306, in _bootstrap
    self.run()
[2018-05-04 18:20:23,222: WARNING/Beat] File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/celery/beat.py", line 613, in run
    self.service.start(embedded_process=True)
[2018-05-04 18:20:23,222: WARNING/Beat] File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/celery/beat.py", line 528, in start
    humanize_seconds(self.scheduler.max_interval))
[2018-05-04 18:20:23,222: WARNING/Beat] File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/kombu/utils/objects.py", line 44, in __get__
    value = obj.__dict__[self.__name__] = self.__get(obj)
[2018-05-04 18:20:23,222: WARNING/Beat] File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/celery/beat.py", line 572, in scheduler
    return self.get_scheduler()
[2018-05-04 18:20:23,222: WARNING/Beat] File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/celery/beat.py", line 567, in get_scheduler
    lazy=lazy,
[2018-05-04 18:20:23,222: WARNING/Beat] File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django_celery_beat/schedulers.py", line 181, in __init__
    Scheduler.__init__(self, *args, **kwargs)
[2018-05-04 18:20:23,223: WARNING/Beat] File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/celery/beat.py", line 204, in __init__
    self.setup_schedule()
[2018-05-04 18:20:23,223: WARNING/Beat] File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django_celery_beat/schedulers.py", line 189, in setup_schedule
    self.install_default_entries(self.schedule)
[2018-05-04 18:20:23,223: WARNING/Beat] File "/Users/amit/thoughtanalytik/koolanch/src/koolanch/beat_schedulers.py", line 27, in schedule
    self._schedule = self.all_as_schedule()
[2018-05-04 18:20:23,223: WARNING/Beat] File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django_celery_beat/schedulers.py", line 195, in all_as_schedule
    for model in self.Model.objects.enabled():
[2018-05-04 18:20:23,223: WARNING/Beat] File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django/db/models/query.py", line 250, in __iter__
    self._fetch_all()
[2018-05-04 18:20:23,224: WARNING/Beat] File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django/db/models/query.py", line 1118, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
[2018-05-04 18:20:23,224: WARNING/Beat] File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django/db/models/query.py", line 53, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch)
[2018-05-04 18:20:23,224: WARNING/Beat] File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 894, in execute_sql
    raise original_exception
[2018-05-04 18:20:23,224: WARNING/Beat] File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 884, in execute_sql
    cursor.execute(sql, params)
[2018-05-04 18:20:23,224: WARNING/Beat] File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django/db/backends/utils.py", line 80, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
[2018-05-04 18:20:23,224: WARNING/Beat] File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
[2018-05-04 18:20:23,224: WARNING/Beat] File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
[2018-05-04 18:20:23,225: WARNING/Beat] File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
[2018-05-04 18:20:23,225: WARNING/Beat] File "/Users/amit/koolanch-dev/dev-1/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
[2018-05-04 18:20:23,225: WARNING/Beat] django.db.utils.ProgrammingError: relation "django_celery_beat_periodictask" does not exist
LINE 1: ...ango_celery_beat_periodictask"."description" FROM "django_ce...

提前致谢。

2 个答案:

答案 0 :(得分:0)

Celery beat(使用django_celery_beat)默认连接到公共模式,并监视django_celery_beat_periodictask表以获取周期性任务。因此,我们必须在公共模式中使用这些表。因此,django_celery_beat app应该是公共模式的一部分,并且必须在公共模式中创建所有周期性任务。

with schema_context('public'):
            schedule, created = IntervalSchedule.objects.get_or_create(
                every=execution_interval,
                period=IntervalSchedule.MINUTES)

            pt_obj, created = PeriodicTask.objects.get_or_create(name=name,
                                                             task='tasks.task_name')
            pt_obj.interval = schedule
            pt_obj.enabled = False
            pt_obj.args = json.dumps([])
            pt_obj.kwargs = json.dumps({})
            pt_obj.save()

答案 1 :(得分:0)

tenant-schemas-celery本身推荐了here所推荐的名为django-tenant-schemas的软件包。