Django,haystack,弹性搜索和一对多关系

时间:2017-05-27 13:58:21

标签: django elasticsearch django-haystack

我有干草堆的问题 - 我不知道如何搜索所有外键符合条件的模型A

我的简化模型如下:

Group:
    id
Meeting:
    group = models.ForeignKey(Group)
    day_of_week = models.IntegerField()
    hour = models.IntegerField()
    length = models.IntegerField()

基本上,一个小组可以召开很多会议,用户应该能够搜索所有会议都在给定时间范围内的小组。例如:

Group(1)
    Meeting(day_of_week=Monday, hour=9, length=2)
Group(2)
    Meeting(day_of_week=Monday, hour=10, length=1)
    Meeting(day_of_week=Tuesday, hour=8, length=2)
Group(3)
    Meeting(day_of_week=Monday, hour=10, length=1)
    Meeting(day_of_week=Wednesday, hour=12, length=1)

并搜索:"星期一从8到11","星期二,从12到14 (下午2点)",&#34 ;星期三,6点到17点(下午5点)"应该返回第1组和第3组,因为来自这些组的所有会议都包含在用户指定的范围内,并且不返回第2组,因为第二次会议不在给定范围内(第一次会议是在第一次会议中)。

如果我要编写SQL,我可能会选择"选择匹配会议的数量和所有会议的数量(如果这些数字相等) - >然后所有会议都会举行:

SELECT g.id,
       count(m2.id)
FROM groups g
JOIN meetings m2 ON m2.group_id = g.id
AND ((m2.day_of_week = 0  -- monday
      AND m2.hour >= 8
      AND m2.length<=3)
     OR (m2.day_of_week=1  -- tuesday
         AND m2.hour >= 12
         AND m2.length<=2)
     OR (m2.day_of_week=2 -- wednesday
         AND m2.hour >= 6
         AND m2.length<=11))
GROUP BY g.id
HAVING count(m2.id) =
  (SELECT count(*)
   FROM meetings
   WHERE meetings.group_id=g.id);

但是我们使用haystack + elastic搜索索引,我完全不知道如何压缩模型来索引和编写查询。任何人都可以帮我吗?

2 个答案:

答案 0 :(得分:1)

您可能需要压缩文档,使所有文档都必须是包含组信息的会议。

** ES 5的解决方案**

您的文档的映射将是:

PUT /meetings
{
    "mappings": {
       "meeting": {
          "properties": {
             "groupId": {
                "type": "integer"
             },
             "dayOfWeek": {
                "type": "integer"
             },
             "hourRange": {
                "type": "integer_range"
             }
          }
       }
    }
}

然后你的五个文件看起来像这样:

POST /meetings/meeting/_bulk
{"index": {}}
{"groupId": 1, "dayOfWeek": 0, "hourRange": {"gte": 9, "lte": 11}}
{"index": {}}
{"groupId": 2, "dayOfWeek": 0, "hourRange": {"gte": 10, "lte": 11}}
{"index": {}}
{"groupId": 2, "dayOfWeek": 1, "hourRange": {"gte": 8, "lte": 10}}
{"index": {}}
{"groupId": 3, "dayOfWeek": 0, "hourRange": {"gte": 10, "lte": 11}}
{"index": {}}
{"groupId": 3, "dayOfWeek": 2, "hourRange": {"gte": 12, "lte": 13}}

最后,查询将如下所示:

POST /meetings/meeting/_search
{
  "query": {
    "bool": {
      "should": [
        {
          "bool": {
            "must": [
              {
                "term": {
                  "dayOfWeek": 0
                }
              },
              {
                "range": {
                  "hourRange": {
                    "gte": "8",
                    "lte": "11",
                    "relation": "within"
                  }
                }
              }
            ]
          }
        },
        {
          "bool": {
            "must": [
              {
                "term": {
                  "dayOfWeek": 1
                }
              },
              {
                "range": {
                  "hourRange": {
                    "gte": "12",
                    "lte": "14",
                    "relation": "within"
                  }
                }
              }
            ]
          }
        },
        {
          "bool": {
            "must": [
              {
                "term": {
                  "dayOfWeek": 2
                }
              },
              {
                "range": {
                  "hourRange": {
                    "gte": "6",
                    "lte": "17",
                    "relation": "within"
                  }
                }
              }
            ]
          }
        }
      ]
    }
  }
}

** ES的解决方案<5 **

PUT /meetings
{
    "mappings": {
       "meeting": {
          "properties": {
             "groupId": {
                "type": "integer"
             },
             "dayOfWeek": {
                "type": "integer"
             },
             "start": {
                "type": "integer"
             },
             "end": {
                "type": "integer"
             }
          }
       }
    }
}

然后你的五个文件看起来像这样:

POST /meetings/meeting/_bulk
{"index": {}}
{"groupId": 1, "dayOfWeek": 0, "start": 9,  "end": 11}
{"index": {}}
{"groupId": 2, "dayOfWeek": 0, "start": 10, "end": 11}
{"index": {}}
{"groupId": 2, "dayOfWeek": 1, "start": 8,  "end": 10}
{"index": {}}
{"groupId": 3, "dayOfWeek": 0, "start": 10, "end": 11}
{"index": {}}
{"groupId": 3, "dayOfWeek": 2, "start": 12, "end": 13}

最后,查询将如下所示:

POST /meetings/meeting/_search
{
  "query": {
    "bool": {
      "should": [
        {
          "bool": {
            "must": [
              {
                "term": {
                  "dayOfWeek": 0
                }
              },
              {
                "range": {
                  "start": {
                    "gte": "8"
                  }
                }
              },
              {
                "range": {
                  "end": {
                    "lte": "11"
                  }
                }
              }
            ]
          }
        },
        {
          "bool": {
            "must": [
              {
                "term": {
                  "dayOfWeek": 1
                }
              },
              {
                "range": {
                  "start": {
                    "gte": "12"
                  }
                }
              },
              {
                "range": {
                  "end": {
                    "lte": "14"
                  }
                }
              }
            ]
          }
        },
        {
          "bool": {
            "must": [
              {
                "term": {
                  "dayOfWeek": 2
                }
              },
              {
                "range": {
                  "start": {
                    "gte": "6"
                  }
                }
              },
              {
                "range": {
                  "end": {
                    "lte": "17"
                  }
                }
              }
            ]
          }
        }
      ]
    }
  }
}

答案 1 :(得分:1)

ElasticSearch解决方案

解决方案的关键是名为nested objects的ElasticSearch功能。幸运的是,所有ES版本都有此功能。嵌套对象在这里是关键,因为会议中的数据是严格关联的。

PUT /myindex
{
  "mappings": {
    "groups": {
      "properties": {
        "meetings": {
          "type": "nested", 
          "properties": {
             "dayOfWeek": { "type": "integer"},
             "start": {"type": "integer"},
             "end": {"type": "integer"}
          }
        },
        "groupId": {"type":"integer"}
      }
    }
  }
}

POST /myindex/groups/_bulk
{"index": {}}
{"groupId": 1, "meetings": [{"dayOfWeek": 0, "start": 9,  "end": 11}]}
{"index": {}}
{"groupId": 2, "meetings": [{"dayOfWeek": 0, "start": 10, "end": 11}, { "dayOfWeek": 1, "start": 8,  "end": 10}]}
{"index": {}}
{"groupId": 3, "meetings": [{"dayOfWeek": 0, "start": 10, "end": 11}, {"dayOfWeek": 2, "start": 12, "end": 13}]}

此时可以清楚地看到会议属于群组,我们将分组搜索。

无法直接编写查询以获取所有嵌套对象满足条件的所有组,但是......可以轻松地将其反转为:获取所有 none 会议中包含错误时间。

GET /myindex/_search
{
  "query": {
    "bool": {
      "must_not" : {
       "nested": {
          "path": "meetings",
          "filter": {
              "bool": {
                "must_not": {
                  "bool": {
                    "should": [
                        {
                          "bool": {
                            "must": [
                              {"term" : { "dayOfWeek" : 0 }},
                              {"range": {"start": {"from":8, "to":11}}},
                              {"range": {"end": {"from":8, "to":11}}}
                            ]
                          }
                        },
                        {
                          "bool": {
                            "must": [
                              {"term" : { "dayOfWeek" : 1 }},
                              {"range": {"start": {"from":12, "to":14}}},
                              {"range": {"end": {"from":12, "to":14}}}
                            ]
                          }
                        },
                        {
                          "bool": {
                            "must": [
                              {"term" : { "dayOfWeek" : 2 }},
                              {"range": {"start": {"from":6, "to":17}}},
                              {"range": {"end": {"from":6, "to":17}}}
                            ]
                          }
                        }                        
                      ]
                    }
                }
              }
            }
          }
       }
      }
    }
  }

这将返回第1组和第3组。由于其中一次会议与错误的日期时间重叠,因此不会返回第2组。

干草堆整合

第二个问题是与Django Haystack的集成,因为默认情况下它不支持引擎特定的功能,如ES中的嵌套字段。幸运的是,我不是唯一一个在django app和someone has already resolved it中需要它的人。