如何根据ManyToManyField过滤Wagtail中的搜索结果?

时间:2017-03-29 00:55:50

标签: django wagtail

我有一个定义Event模型的Wagtail网站。这些事件有多个事件赞助商,它们与ManyToManyField模型上的EventSponsor相关联:

class Event(index.Indexed, ClusterableModel):

    title       = models.CharField(max_length=255)
    start_date  = models.DateTimeField()
    end_date    = models.DateTimeField(null=True, blank=True)
    description = RichTextField(blank=True)

    search_fields = [
        index.SearchField('title', partial_match=True, boost=2.0),
        index.SearchField('description'),
        index.RelatedFields('sponsors', [
            index.SearchField('name', partial_match=True)
        ]),

        index.FilterField('end_date'),
        index.FilterField('sponsors'),
    ]

class EventSponsor(index.Indexed, models.Model):

    sponsor_id = models.IntegerField()
    name = models.CharField(max_length=255)
    url = models.URLField(blank=True)

    events = models.ManyToManyField(Event, related_name='sponsors')

    search_fields = [
        index.SearchField('name', partial_match=True),
    ]

除此之外,我的Wagtail服务器上的不同站点在其日历中包含基于特定于该站点的一组选定事件赞助商的事件。

因此,为每个站点构建日历列表​​查询集如下所示:

def get_events_for_current_site(request, listing):
    try:
        event_sponsor_settings = EventSponsorSettings.objects.get(site=request.site)
    except EventSponsorSettings.DoesNotExist:
        # If there's no EventSponsorSettings for this Site, return an empty QuerySet. This shouldn't really ever happen.
        return Event.objects.none()

    # Return the selected Events in decending order of start date.
    query = Event.objects.filter(sponsors__in=event_sponsor_settings.selected_event_sponsors)
    if listing == 'upcoming_events':
        return query.order_by('start_date').filter(end_date__gte=timezone.now())
    else:
        return query.order_by('-start_date').filter(end_date__lt=timezone.now())

event_sponsor_settings.selected_event_sponsorsEventSponsor个对象的列表。此查询集适用于列表页面。

我需要在每个网站上使用搜索功能(使用Elasticsearch后端),以仅包含将出现在当前网站日历上的事件。所以我希望我的基本查询集与日历页面使用的相同(或者至少进行相同的过滤)。所以我的事件搜索代码基本上叫:

backend.search(search_query, get_events_for_current_site())

但是,我遇到了两个问题:

1)如果我在index.FilterField('sponsors')中使用Event.search_fields,则在运行manage.py update_index时出现此错误:

Traceback (most recent call last):
  File "./manage.py", line 33, in <module>
    execute_from_command_line(argv)
  File "/multitenant-ve/lib/python2.7/site-packages/django/core/management/__init__.py", line 353, in execute_from_command_line
    utility.execute()
  File "/multitenant-ve/lib/python2.7/site-packages/django/core/management/__init__.py", line 345, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/multitenant-ve/lib/python2.7/site-packages/django/core/management/base.py", line 348, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/multitenant-ve/lib/python2.7/site-packages/django/core/management/base.py", line 399, in execute
    output = self.handle(*args, **options)
  File "/multitenant-ve/src/wagtail/wagtail/wagtailsearch/management/commands/update_index.py", line 120, in handle
    self.update_backend(backend_name, schema_only=options.get('schema_only', False))
  File "/multitenant-ve/src/wagtail/wagtail/wagtailsearch/management/commands/update_index.py", line 77, in update_backend
    index.add_model(model)
  File "/multitenant-ve/src/wagtail/wagtail/wagtailsearch/backends/elasticsearch.py", line 536, in add_model
    index=self.name, doc_type=mapping.get_document_type(), body=mapping.get_mapping()
  File "/multitenant-ve/src/wagtail/wagtail/wagtailsearch/backends/elasticsearch.py", line 137, in get_mapping
    self.get_field_mapping(field) for field in self.model.get_search_fields()
  File "/multitenant-ve/src/wagtail/wagtail/wagtailsearch/backends/elasticsearch.py", line 137, in <genexpr>
    self.get_field_mapping(field) for field in self.model.get_search_fields()
  File "/multitenant-ve/src/wagtail/wagtail/wagtailsearch/backends/elasticsearch.py", line 119, in get_field_mapping
    return self.get_field_column_name(field), mapping
  File "/multitenant-ve/src/wagtail/wagtail/wagtailsearch/backends/elasticsearch.py", line 72, in get_field_column_name
    return field.get_attname(self.model) + '_filter'
  File "/multitenant-ve/src/wagtail/wagtail/wagtailsearch/index.py", line 178, in get_attname
    return field.attname
AttributeError: 'ManyToManyRel' object has no attribute 'attname'

2)如果我取出index.FilterField('sponsors')manage.py update_index有效,但我搜索时出错:

Cannot filter search results with field "eventsponsor_id". Please add index.FilterField('eventsponsor_id') to Event.search_fields.

所以我尝试添加index.FilterField('eventsponsor_id'),它会在update_indexEvent.search_fields contains field 'eventsponsor_id' but it doesn't exist期间发出此警告,并在搜索时导致此追溯:

Traceback:
File "/multitenant-ve/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  174.                     response = self.process_exception_by_middleware(e, request)
File "/multitenant-ve/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  172.                     response = response.render()
File "/multitenant-ve/lib/python2.7/site-packages/django/template/response.py" in render
  160.             self.content = self.rendered_content
File "/multitenant-ve/lib/python2.7/site-packages/django/template/response.py" in rendered_content
  137.         content = template.render(context, self._request)
File "/multitenant-ve/lib/python2.7/site-packages/django/template/backends/django.py" in render
  95.             return self.template.render(context)
File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in render
  206.                     return self._render(context)
File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in _render
  197.         return self.nodelist.render(context)
File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in render
  992.                 bit = node.render_annotated(context)
File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in render_annotated
  959.             return self.render(context)
File "/multitenant-ve/lib/python2.7/site-packages/django/template/loader_tags.py" in render
  173.         return compiled_parent._render(context)
File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in _render
  197.         return self.nodelist.render(context)
File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in render
  992.                 bit = node.render_annotated(context)
File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in render_annotated
  959.             return self.render(context)
File "/multitenant-ve/lib/python2.7/site-packages/django/template/loader_tags.py" in render
  173.         return compiled_parent._render(context)
File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in _render
  197.         return self.nodelist.render(context)
File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in render
  992.                 bit = node.render_annotated(context)
File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in render_annotated
  959.             return self.render(context)
File "/multitenant-ve/lib/python2.7/site-packages/django/template/loader_tags.py" in render
  69.                 result = block.nodelist.render(context)
File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in render
  992.                 bit = node.render_annotated(context)
File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in render_annotated
  959.             return self.render(context)
File "/multitenant-ve/lib/python2.7/site-packages/django/template/defaulttags.py" in render
  220.                     nodelist.append(node.render_annotated(context))
File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in render_annotated
  959.             return self.render(context)
File "/multitenant-ve/lib/python2.7/site-packages/django/template/defaulttags.py" in render
  325.             if match:
File "/multitenant-ve/src/wagtail/wagtail/wagtailsearch/backends/base.py" in __len__
  174.         return len(self.results())
File "/multitenant-ve/src/wagtail/wagtail/wagtailsearch/backends/base.py" in results
  137.             self._results_cache = self._do_search()
File "/multitenant-ve/src/wagtail/wagtail/wagtailsearch/backends/elasticsearch.py" in _do_search
  452.         hits = self.backend.es.search(**params)
File "/multitenant-ve/lib/python2.7/site-packages/elasticsearch/client/utils.py" in _wrapped
  69.             return func(*args, params=params, **kwargs)
File "/multitenant-ve/lib/python2.7/site-packages/elasticsearch/client/__init__.py" in search
  531.             doc_type, '_search'), params=params, body=body)
File "/multitenant-ve/lib/python2.7/site-packages/elasticsearch/transport.py" in perform_request
  273.             body = self.serializer.dumps(body)
File "/multitenant-ve/lib/python2.7/site-packages/elasticsearch/serializer.py" in dumps
  47.             raise SerializationError(data, e)

Exception Type: SerializationError at /search
Exception Value: ({u'query': {u'filtered': {u'filter': {u'and': [{u'prefix': {u'content_type': u'event'}}, {'and': [{u'terms': {u'eventsponsor_id_filter': [<EventSponsor: Division of Geological and Planetary Sciences (9003)>]}}, {u'range': {u'end_date_filter': {'gte': datetime.datetime(2017, 3, 29, 0, 42, 7, 462939, tzinfo=<UTC>)}}}]}]}, u'query': {u'multi_match': {u'query': u'geo', u'fields': [u'_all', u'_partials']}}}}}, TypeError("Unable to serialize <EventSponsor: Division of Geological and Planetary Sciences (9003)> (type: <class 'templated_cms.models.events.EventSponsor'>)",))

因此,我尝试将get_events_for_current_site()中的查询集更改为Event.objects.filter(sponsors__id__in=[s.id for s in event_sponsor_settings.selected_event_sponsors])

这可以修复错误......但我根本没有搜索结果。

我完全不知道如何处理这个问题。 :(

3 个答案:

答案 0 :(得分:2)

对于初学者来说,这篇文章帮助我解决了这个问题,所以非常感谢你。

FilterFields非常适合在搜索结果上运行过滤器。在这种情况下,我们只需要从过滤后的查询集中构建搜索结果。

我的方法如下:

  1. 收集您想要形成搜索结果的事件的ID。

    event_ids = get_events_for_current_site().values_list('id', flat=True)
    
  2. 根据这些ID构建新的查询集。

    filtered_events = Event.objects.filter(id__in=event_ids)
    
  3. 将新的查询集传递给您的搜索

    backend.search(search_query, filtered_events)
    
  4. 由于传入搜索的查询集已从ID中过滤掉,因此您需要在index.FilterField('id')中加入Event.search_fields并更新索引。

    请注意,我没有专门测试报告的代码,而是我自己的变体。

    此外,这篇Wagtail Support帖子让我对解决这个问题有了一些见解:https://groups.google.com/forum/#!msg/wagtail/k2-E4h2oLtI/uPOzbuwKBgAJ

    这篇文章确实有一个警告说,只要你没有1000件[项目],使用这种方法“不应该打得太糟糕”。

答案 1 :(得分:1)

我在搜索/过滤表单中的文本和许多字段方面也遇到类似的问题。

我解决它的方式:

  1. 对模型执行ElasticSearch。
  2. 将结果(DatabaseSearchResults)转换为查询集。
  3. 将许多过滤器应用于来自表单的数据的查询集

例如

results = MyModel.search(search_terms, fields=['title', 'body'], operator='or')
qs = results.get_queryset()

m2m_objects = self.cleaned_data.get('m2m_field')
qs = qs.filter(m2m_field__in=m2m_objects)

答案 2 :(得分:0)

对于那些在将来遇到这个问题的人来说,这就是我最终解决这个问题的方法(代码已被削减到显示我使用的机制的最低限度):

class Event(index.Indexed, ClusterableModel):

    title       = models.CharField(max_length=255)
    start_date  = models.DateTimeField()
    end_date    = models.DateTimeField(null=True, blank=True)
    description = RichTextField(blank=True)
    lecture_series = models.ForeignKey(
        'this_app.LectureSeries', null=True, blank=True, related_name='events', 
        on_delete=models.SET_NULL
    )

    search_fields = [
        ...
        # We use a Filterfield on lecture_series here because we apparently can't do it 
        # on lecture_series_id for whatever reason. This means we need to filter Events
        # on their lecture_series directly on all querysets that will get used as a 
        # search filter.
        index.FilterField('lecture_series'),
        # We can't filter directly on a ManyToMany relationship, so we need to be a bit
        # creative. This uses the sponsor_id() method defined below to add our 
        # EventSponsors' sponsor_ids to the search index.
        index.FilterField('sponsor_id'),
    ]

    def sponsor_id(self):
        """
        Adds all of our EventSponsors' sponsor_ids to the search filter list.
        """
        return list(self.sponsors.all().values_list('sponsor_id', flat=True))


class EventSponsor(index.Indexed, models.Model):

    sponsor_id = models.IntegerField()
    name = models.CharField(max_length=255)

    events = models.ManyToManyField(Event, related_name='sponsors')

    search_fields = [
        index.SearchField('name', partial_match=True),
    ]


class LectureSeries(index.Indexed, models.Model):

    lecture_series_id = models.IntegerField(unique=True)
    name = models.CharField(max_length=255)

    search_fields = [
        index.SearchField('name', partial_match=True),
    ]


def get_base_events_queryset_for_site(site):
    """
    Returns the base queryset object from which all Event listings for a spcified Site
    must be derived.
    This function filters the list of Event objects down to just those that the Site's
    admins have chosen to display.
    """
    try:
        settings = EventListingSettings.objects.get(site=site)
    except EventListingSettings.DoesNotExist:
        # If there's no EventListingSettings for this Site, return an empty QuerySet.
        return Event.objects.none()

    # We need to do the sponsors via their sponsor_ids because searches can't be filtered
    # directly on a ManyToMany relationship.
    sponsor_ids = [sponsor.sponsor_id for sponsor in settings.event_sponsors.all()]

    # We need to split these up for Sites which import either no LectureSeries or no 
    # EventSponsors. Listings will get dupes, and searches will crash if we don't.
    if settings.lecture_series.exists() and sponsor_ids:
        queryset = Event.objects.filter(
            Q(sponsors__sponsor_id__in=sponsor_ids) | 
            Q(lecture_series__in=settings.lecture_series.all())
        )
    elif sponsor_ids:
        queryset = event_model.objects.filter(sponsors__sponsor_id__in=sponsor_ids)
    else:
        queryset = event_model.objects.filter(lecture_series__in=settings.lecture_series.all())

    return queryset

正如您所看到的,常规ForeignKeys可以像往常一样用于过滤搜索,但ManyToMany关系需要一些特殊的ID列表代码,以便可以构建可以转换为ElasticSearch查询的查询集。