使用大型数据集加速django测试

时间:2014-03-08 13:06:48

标签: python django transactional django-unittest

我在django中使用大型数据集进行非常慢的单元测试运行(2小时+)时遇到了问题,所以写下以下内容以实现更快的测试,通过替换所有的夹具加载将时间缩短到40分钟我最初编写的用静态django模型对象替换我们的灯具的测试创建。

from django.core.management import call_command
from django.db import transaction, DEFAULT_DB_ALIAS, connection, connections
from django import test
from django.test import TransactionTestCase
from django.test.testcases import disable_transaction_methods, \
    restore_transaction_methods


class FastTestCase(TransactionTestCase):
    """
    FastTestCase is a TransactionTestCase utilising a save point following the global setup
    to rollback following each test. This works in the same way FastFixtures would work,
    but allows rolling back to after the fixtures were loaded in order to speed up tests when
    very large data sets are being created.
    FastTestCase was written to assist in avoiding having to make a lot of changes to JSON fixtures
    when model changes had been made in order to enable unit tests to continue passing.
    NOTE: this is slower when small test data sets are being used!
    """

    @classmethod
    def setUpClass(cls):
        if test.testcases.connections_support_transactions():
            transaction.enter_transaction_management(using=DEFAULT_DB_ALIAS)
            transaction.managed(True, using=DEFAULT_DB_ALIAS)
            disable_transaction_methods()
            cls.fixtures_loaded = False

    @classmethod
    def tearDownClass(cls):
        if test.testcases.connections_support_transactions():
            restore_transaction_methods()
            transaction.rollback(using=DEFAULT_DB_ALIAS)
            transaction.leave_transaction_management(using=DEFAULT_DB_ALIAS)
        for connection in connections.all():
            connection.close()

    def globalSetUp(self):
        """
        Hook method to be overridden and provide fast fixtures style of setup
        """

    def _fixture_setup(self):
        from django.contrib.sites.models import Site
        Site.objects.clear_cache()

        if not self.__class__.fixtures_loaded:
            if hasattr(self, 'fixtures'):
                call_command('loaddata',*self.fixtures, **{'verbosity':0,'commit':False,'database':DEFAULT_DB_ALIAS})
            self.globalSetUp()
            if test.testcases.connections_support_transactions():
                self.__class__.fixtures_loaded = True
        if test.testcases.connections_support_transactions():
            self.save_point = transaction.savepoint()

    def _fixture_teardown(self):
        if test.testcases.connections_support_transactions():
            transaction.savepoint_rollback(self.save_point)
        else:
            super(FastTestCase, self)._fixture_teardown()

    def _post_teardown(self):
        self._fixture_teardown()
        self._urlconf_teardown()

我现在正在寻求进一步提高速度是否有任何方法可以让我更快和更快对较小的数据集有效吗?

0 个答案:

没有答案