如何加速使用.distinct()的查询集?

时间:2013-08-29 17:52:30

标签: django performance postgresql django-models postgresql-9.1

我希望加快需要使用distinct的查询,因为它有一个选择的M2M字段。此时我不确定我的速度问题是否与我如何配置数据库服务器或者它与我的查询集有关。

我的问题:什么是最快的查询集,我还可以通过更改Postgresql设置来提高速度吗?

Postgresql服务器信息

实例:EC2 m1.xlarge
Postgresql版本:9.1
文章记录:240,695
总内存:14980 MB
shared_buffers:3617MB
effective_cache_size:8000MB
work_mem:40MB
checkpoint_segments:10
maintenance_work_mem:64MB

相关模型

class AuthorsModelMixin(models.Model):

    authors = models.ManyToManyField('people.Person', blank=True)
    nonstaff_authors = models.CharField(
        verbose_name='Non-staff authors', max_length=255, blank=True,
        help_text="Used for the name of the author for non-staff members.")
    byline_title = models.CharField(
        max_length=255, blank=True,
        help_text="Often contains an organization. Title of the person, or " \
                  "entity associated with the byline and a specified person " \
                  "(i.e. Associated Press).")

    class Meta:
        abstract = True

class TaxonomyModelMixin(models.Model):

    sections = models.ManyToManyField(Section, blank=True)
    tags = TaggableManager(
        blank=True, help_text='A comma-separated list of tags (i.e. ' \
                              'Outdoors, Election, My Great News Topic).')

    class Meta:
        abstract = True

class PublishModelMixin(models.Model):

    status_choices = (
        ('D', 'Draft'),
        ('P', 'Published'),
        ('T', 'Trash'),
    )

    comment_choices = (
        ('enabled', 'Enabled'),
        ('disabled', 'Disabled'),
    )

    sites = models.ManyToManyField(Site, default=[1])
    status = models.CharField(
        max_length=1, default='P', db_index=True, choices=status_choices,
        help_text='Only published items will appear on the site')
    published = models.DateTimeField(
        default=timezone.now, db_index=True,
        help_text='Select the date you want the content to be published.')
    is_premium = models.BooleanField(
        choices=((True, 'Yes'), (False, 'No')),
        verbose_name='Premium Content', default=True)
    comments = models.CharField(
        max_length=30, default='enabled',
        choices=comment_choices, help_text='Enable or disable comments.')
    created = models.DateTimeField(auto_now_add=True)
    modified = models.DateTimeField(auto_now=True)
    objects = PublishedManager()

    class Meta:
        abstract = True

class Article(AuthorsModelMixin, TaxonomyModelMixin, PublishModelMixin):

    title = models.CharField(max_length=255)
    slug = SlugModelField(max_length=255)
    lead_photo = models.ForeignKey('media.Photo', blank=True, null=True)
    summary = models.TextField(blank=True)
    body = models.TextField()

我尝试过的查询集

Queryset 1

查询时间:(76毫秒)
优点:快速,没有机会发表文章不会显示
缺点:如果较高的ID具有较旧的发布日期,则文章列表将无序

queryset = Article.objects \
                  .published() \
                  .filter(sections__full_slug__startswith=section.full_slug) \
                  .prefetch_related('lead_photo', 'authors') \
                  .order_by('-id') \
                  .distinct('id')

Queryset 2

查询时间:(76毫秒)
优点:文章始终有序 缺点:如果两篇文章具有相同的发布日期和时间,则只列出一个

queryset = Article.objects \
                  .published() \
                  .filter(sections__full_slug__startswith=section.full_slug) \
                  .prefetch_related('lead_photo', 'authors') \
                  .order_by('-published') \
                  .distinct('published')

Queryset 3

查询时间:(1007毫秒)
优点:文章一直在按顺序排列,没有文章没有被列入的机会 缺点:慢得多!

queryset = Article.objects \
                  .published() \
                  .filter(sections__full_slug__startswith=section.full_slug) \
                  .prefetch_related('lead_photo', 'authors') \
                  .order_by('-id', '-published') \
                  .distinct('id')

Queryset 4

查询时间:(4797.85 ms)
优点:不多,但不使用DISTINCT ON意味着它适用于SQLite等其他数据库进行测试 缺点:慢得多!!!

queryset = Article.objects \
                  .published() \
                  .filter(sections__full_slug__startswith=section.full_slug) \
                  .prefetch_related('lead_photo', 'authors') \
                  .order_by('-published') \
                  .distinct()

1 个答案:

答案 0 :(得分:2)

您可以尝试对此查询进行性能测试吗?由于您尚未发布模型,请调整任何字段名称。

我的想法是把它分成两个:一个将返回查看中间表的所有文章ID。

queryset = Article.objects \
    .published() \
    .filter(id__in=Article.sections.through.objects
            .filter(section__full_slug__startswith=section.full_slug)
            .values_list('article_id', flat=True)) \
    .prefetch_related('lead_photo', 'authors') \
    .order_by('-published', '-id')