如何从django中的大型数据库加速数据迁移

时间:2015-03-09 13:49:51

标签: django performance django-models

我尝试从我的django 1.7应用程序中删除模型继承

models.py

class Element(models.Model):
    ELEMENT_A = 'ELEMENT_A'
    ELEMENT_B = 'ELEMENT_B'
    TYPES = (
        (ELEMENT_A, 'Element A'),
        (ELEMENT_B, 'Element B')
    )
    number = models.CharField(max_length=255)
    type = models.CharField(max_length=10, choices=TYPES, default=ELEMENT_A)
    quantity = models.IntegerField(default=1)

class ChildElement(Element):
    old_number = models.CharField(max_length=30, unique=True)
    old_quantity = models.IntegerField()

我的迁移文件

def forwards_remove_heritance(apps, schema_editor):
    childs = apps.get_model("core", "ChildElement")
    type = Element.ELEMENT_A
    for c in childs.objects.all():
        c.number = c.old_number
        c.quantity = c.old_quantity
        c.type = type
        c.save()
    return


def backward_remove_heritance(apps, schema_editor):
    return


class Migration(migrations.Migration):
    dependencies = [

    ]

    operations = [
        migrations.RunPython(
            forwards_remove_heritance, backward_remove_heritance
        ),
    ]

我的迁移需要几个小时,我在app_ChildElement表中有超过750k个元素

有什么想法吗?

2 个答案:

答案 0 :(得分:0)

将queryset updateF expressions一起使用,例如:

ChildElement.objects.all().update(number=F('old_number'))

答案 1 :(得分:0)

我找到的解决方案是直接使用操作中的原始SQL更新我的数据库

class RemoveChild(Operation):
    reversible = True

    def __init__(self):
        pass

    def state_forwards(self, app_label, state):
        pass

    def database_forwards(self, app_label, schema_editor, from_state, to_state):
        schema_editor.execute("""update Element
SET Element.number = ChildElement.old_number, Element.quantity=ChildElement.old_quantity, Element.type='ELEMENT_B'
FROM Element
INNER JOIN ChildElement
ON ChildElement.element_ptr_id = Element.id;
""")

    def database_backwards(self, app_label, schema_editor, from_state, to_state):
        pass

    def describe(self):
        return "Remove child heritance model"