Question

我正在编写Django命令，以从我的应用中删除早于x天的数据。

使用以下内容进行过滤：

qs = Data.objects.filter(date_created__lte=timezone.now()-timedelta(days=days_del))

days_del是整数，date_created是DateTimeField。

当尝试打印此查询集或对其调用.delete()时，将得到JSONDecodeError和ValidationError。我真的不知道为什么会发生这种情况，或者在这种情况下我无法阻止它尝试解码JSON文件。

请注意，我正在使用jsonfield pypi包，并且数据模型具有JSONField。

某些数据可能会停滞并导致问题（请参阅回溯），有没有办法忽略验证并继续删除呢？

Traceback (most recent call last):
  File "/root/virtualenvs/myproj-prod/lib/python3.6/site-packages/jsonfield/fields.py", line 83, in pre_init
    return json.loads(value, **self.load_kwargs)
  File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.6/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 464 (char 463)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/root/virtualenvs/myproj-prod/lib/python3.6/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/root/virtualenvs/myproj-prod/lib/python3.6/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/root/virtualenvs/myproj-prod/lib/python3.6/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/root/virtualenvs/myproj-prod/lib/python3.6/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/webapps/myproj/server/mirrors/management/commands/data_cleanup.py", line 39, in handle
    qs.delete()
  File "/root/virtualenvs/myproj-prod/lib/python3.6/site-packages/django/db/models/query.py", line 616, in delete
    collector.collect(del_query)
  File "/root/virtualenvs/myproj-prod/lib/python3.6/site-packages/django/db/models/deletion.py", line 191, in collect
    reverse_dependency=reverse_dependency)
  File "/root/virtualenvs/myproj-prod/lib/python3.6/site-packages/django/db/models/deletion.py", line 89, in add
    if not objs:
  File "/root/virtualenvs/myproj-prod/lib/python3.6/site-packages/django/db/models/query.py", line 254, in __bool__
    self._fetch_all()
  File "/root/virtualenvs/myproj-prod/lib/python3.6/site-packages/django/db/models/query.py", line 1118, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/root/virtualenvs/myproj-prod/lib/python3.6/site-packages/django/db/models/query.py", line 63, in __iter__
    obj = model_cls.from_db(db, init_list, row[model_fields_start:model_fields_end])
  File "/root/virtualenvs/myproj-prod/lib/python3.6/site-packages/django/db/models/base.py", line 583, in from_db
    new = cls(*values)
  File "/root/virtualenvs/myproj-prod/lib/python3.6/site-packages/django/db/models/base.py", line 502, in __init__
    _setattr(self, field.attname, val)
  File "/root/virtualenvs/myproj-prod/lib/python3.6/site-packages/jsonfield/subclassing.py", line 43, in __set__
    obj.__dict__[self.field.name] = self.field.pre_init(value, obj)
  File "/root/virtualenvs/myproj-prod/lib/python3.6/site-packages/jsonfield/fields.py", line 85, in pre_init
    raise ValidationError(_("Enter valid JSON"))
django.core.exceptions.ValidationError: ['Enter valid JSON']

我要同时删除很多数据，也许有更好的方法来处理。无论如何，这里不能修复旧的数据。

谢谢

这是命令文件：

from django.core.management.base import BaseCommand, CommandError
from oauth2_provider.models import Application
from django.utils import timezone

import pytz
from datetime import timedelta

from confluence_core.models import Data



class Command(BaseCommand):
    help = 'Delete data older than given days'

    def add_arguments(self, parser):
        parser.add_argument("-d", "--days", type=int, dest='days', required=True, help="Days limit")
        parser.add_argument("-c", "--confirm", action='store_true', dest='confirm', default=False, required=False, help="Confirm before deletion")

    def handle(self, *args, **options):

        days_del = options['days']
        do_delete = False

        qs = Data.objects.filter(date_created__lte=timezone.now()-timedelta(days=days_del))

        if qs.count() > 0:

            if options['confirm']:
                print(f"{qs.count():,} data entries will be deleted.")
                ret = input("Confirm ? (y/n)\n")
                if ret in ['y', 'Y', 'yes']:
                    do_delete = True
            else:
                do_delete = True

            if do_delete is True:
                print(f"Deleting {qs.count():,} data entries...")
                qs.delete()
            else:
                print("Not deleting anything.")

        else:
            print("No data to delete.")

        print("Done.")

Answer 1

鉴于删除时ORM会强制加载查询集，请参见：

File "/root/virtualenvs/myproj-prod/lib/python3.6/site-packages/django/db/models/deletion.py", line 89, in add
if not objs:

我能想到的唯一解决方法（不需要任何派生或monkeypatching的工作）将是首先更新整个查询集，以便将jsonfield设置为有效值，即：

qs.update(name_of_the_field={})
qs.delete()

但这不会阻止数据不一致的其他问题，因此，真正的解决方案显然是清理整个数据集。

删除queryset时的Django ValidationError

1 个答案: