Elasticsearch过滤器仅显示包含列表值的文档

时间:2014-08-14 13:35:47

标签: elasticsearch

使用以下脚本我设置elasticsearch来索引文章。每篇文章都有一份作者列表:

#!/usr/bin/env bash 
HOST='http://localhost:9200/articles'

curl -XPUT ${HOST} 
curl -XPUT "${HOST}/article/_mapping" -d '{
    "article" : {
        "properties" : {
            "authors" : {"type":"string", "index_name":"author", "index": "not_analyzed"}
        }
    } 
}' 
curl -XPOST  "${HOST}/article/1"  -d '{
    "authors" : ["Albert","Wolfgang","Richard","Murray"],
    "message" : "Blabla" }' 
curl -XPOST  "${HOST}/article/2"  -d '{
    "authors" : ["Albert","Richard"],
    "message" : "Blublu" }' 
curl -XPOST  "${HOST}/article/3"  -d '{
    "authors" : ["Albert"],
    "message" : "Bleble" }'

我想要做的是过滤掉所有不在给定列表中的作者的文章。我尝试了以下查询:

curl -XGET  "${HOST}/_search?pretty=true" -d '{
    "query": {
        "constant_score": {
            "filter": {
                "terms": { 
                    "authors": ["Albert","Richard","Erwin"],
                    "execution": "or" 
                }            
            }
        }
    } 
}'

然而,这将返回所有三篇文章作为点击。但我确实希望过滤掉第1条,因为它有一些作者["Wolfgang", "Murray"],这些作者不在给定作者["Albert", "Richard","Erwin"]的列表中。这可以用弹性搜索来实现吗?

2 个答案:

答案 0 :(得分:1)

棘手的一个。如果我理解得很清楚,您需要查找作者字段中仅包含 这三个值中的一个(或更多)的文档

根据elasticsearch权威指南here,检查字段是否包含 某些值非常昂贵。

有一个丑陋的解决方案受到equality paragraph的启发,它会使用bool过滤器的组合和添加author_count字段。

您必须检查是否:

  • 作者包含Albert / Richard / Erwin之一,并且author_count等于1
  • 作者包含2位作者(Albert& Richard,Erwin& Albert等...)的可能组合之一,其author_count为2
  • 作者包含3位作者,其author_count为3

这对我来说不可行。

答案 1 :(得分:1)

正如Tom83所建议的那样,在弹性搜索权威指南链接中,这可以通过在索引时添加authors_counts属性来完成。像这样:

curl -XPOST  "${HOST}/article/1"  -d '{
  "authors": [
    "Albert",
    "Wolfgang",
    "Richard",
    "Murray"
  ],
  "authors_counts": 4,
  "message": "Blabla"
}'

curl -XPOST  "${HOST}/article/2"  -d '{
  "authors": [
    "Albert",
    "Richard"
  ],
  "authors_counts": 2,
  "message": "Blublu"
}'

curl -XPOST  "${HOST}/article/3"  -d '{
  "authors": [
    "Albert"
  ],
  "authors_counts": 1,
  "message": "Bleble"
}'

然后,您可以按照指南中的建议创建查询。它非常冗长,所以我决定生成它。

(ns query-gen 
  (:require [clojure.data.json :as json]
            [clojure.math.combinatorics :refer [subsets]]))

(defn gen-filter [items]
  (let [terms (map (fn [term] { "term" { "authors" term } }) items)
        terms_count { "term" { "authors_counts" (count items) }}] 
      { "bool" { "must" (cons terms_count terms)}}))

(defn gen-query [names]
  (let [subsets (rest (subsets names))
        filters (map gen-filter subsets)]
    {"query" { "filtered" { "filter" { "or" filters }}}}))

(defn -main [& args]
  (let [ query (gen-query ["Albert" "Richard" "Erwin"])
         json (json/write-str query)]
      (println json)))

这会生成如下所示的查询:

{
  "query": {
    "filtered": {
      "filter": {
        "or": [
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 1
                  }
                },
                {
                  "term": {
                    "authors": "Albert"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 1
                  }
                },
                {
                  "term": {
                    "authors": "Richard"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 1
                  }
                },
                {
                  "term": {
                    "authors": "Erwin"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 2
                  }
                },
                {
                  "term": {
                    "authors": "Albert"
                  }
                },
                {
                  "term": {
                    "authors": "Richard"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 2
                  }
                },
                {
                  "term": {
                    "authors": "Albert"
                  }
                },
                {
                  "term": {
                    "authors": "Erwin"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 2
                  }
                },
                {
                  "term": {
                    "authors": "Richard"
                  }
                },
                {
                  "term": {
                    "authors": "Erwin"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 3
                  }
                },
                {
                  "term": {
                    "authors": "Albert"
                  }
                },
                {
                  "term": {
                    "authors": "Richard"
                  }
                },
                {
                  "term": {
                    "authors": "Erwin"
                  }
                }
              ]
            }
          }
        ]
      }
    }
  }
}

如果使用如下:

curl -XGET  "${HOST}/_search?pretty=true" -d '{
  "query": {
    "filtered": {
      "filter": {
        "or": [
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 1
                  }
                },
                {
                  "term": {
                    "authors": "Albert"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 1
                  }
                },
                {
                  "term": {
                    "authors": "Richard"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 1
                  }
                },
                {
                  "term": {
                    "authors": "Erwin"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 2
                  }
                },
                {
                  "term": {
                    "authors": "Albert"
                  }
                },
                {
                  "term": {
                    "authors": "Richard"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 2
                  }
                },
                {
                  "term": {
                    "authors": "Albert"
                  }
                },
                {
                  "term": {
                    "authors": "Erwin"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 2
                  }
                },
                {
                  "term": {
                    "authors": "Richard"
                  }
                },
                {
                  "term": {
                    "authors": "Erwin"
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must": [
                {
                  "term": {
                    "authors_counts": 3
                  }
                },
                {
                  "term": {
                    "authors": "Albert"
                  }
                },
                {
                  "term": {
                    "authors": "Richard"
                  }
                },
                {
                  "term": {
                    "authors": "Erwin"
                  }
                }
              ]
            }
          }
        ]
      }
    }
  }
}'

它会返回我认为您期望的结果:

{
  "took": 5,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 1.0,
    "hits": [
      {
        "_index": "articles",
        "_type": "article",
        "_id": "3",
        "_score": 1.0,
        "_source": {
          "authors": [
            "Albert"
          ],
          "authors_counts": 1,
          "message": "Bleble"
        }
      },
      {
        "_index": "articles",
        "_type": "article",
        "_id": "2",
        "_score": 1.0,
        "_source": {
          "authors": [
            "Albert",
            "Richard"
          ],
          "authors_counts": 2,
          "message": "Blublu"
        }
      }
    ]
  }
}

我不确定这是不是一个好主意,但非常有趣。希望它有所帮助!