Elasticsearch-在多个字段上聚合,按数量过滤和按数量排序

时间:2019-03-13 08:44:36

标签: elasticsearch elasticsearch-aggregation

我对聚合有点陌生,我想创建一个等效于以下SQL的

select fullname, natcode, count(1) from table where birthdate = '18-sep-1993' group by fullname, natcode having count(1) > 2 order by count(1) desc

所以,如果我有以下数据:
enter image description here

我需要得到如下结果:
enter image description here

如您所见,结果按全名和natcode分组,计数为2,并按计数排序

我设法形成以下查询:

{
  "size": 0,
  "aggs": {
    "profs": {
      "filter": {
        "term": {
          "birthDate": "18-Sep-1993"
        }
      },
      "aggs": {
        "name_count": {
          "terms": {
            "field": "fullName.raw"
          },
          "aggs": {
            "nat_count": {
              "terms": {
                "field": "natCode"
              },
              "aggs": {
                "my_filter": {
                  "bucket_selector": {
                    "buckets_path": {
                      "the_doc_count": "_count"
                    },
                    "script": {
                      "source": "params.the_doc_count>2"
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

实现的目标: 它按日期进行过滤,对全名(name_count)创建存储桶,对natcode(nat_count)创建子存储桶,并根据文档计数过滤natcode存储桶。

与此有关的问题: 我也可以看到空的name_count存储桶。我只想要具有所需数量的水桶。以下是结果示例

"aggregations": {
    "profs": {
      "doc_count": 3754,
      "name_count": {
        "doc_count_error_upper_bound": 4,
        "sum_other_doc_count": 3732,
        "buckets": [
          {
            "key": "JOHN SMITH",
            "doc_count": 3,
            "nat_count": {
              "doc_count_error_upper_bound": 0,
              "sum_other_doc_count": 0,
              "buckets": [
                {
                  "key": "111",
                  "doc_count": 3
                }
              ]
            }
          },
          {
            "key": "MIKE CAIN",
            "doc_count": 3,
            "nat_count": {
              "doc_count_error_upper_bound": 0,
              "sum_other_doc_count": 0,
              "buckets": [
                {
                  "key": "205",
                  "doc_count": 3
                }
              ]
            }
          },
          {
            "key": "JULIA ROBERTS",
            "doc_count": 2,
            "nat_count": {
              "doc_count_error_upper_bound": 0,
              "sum_other_doc_count": 0,
              "buckets": []
            }
          },
          {
            "key": "JAMES STEPHEN COOK",
            "doc_count": 2,
            "nat_count": {
              "doc_count_error_upper_bound": 0,
              "sum_other_doc_count": 0,
              "buckets": []
            }
          }

在结果中,我不希望显示最后两个名字(JULIA ROBERTS和JAMES STEPHEN COOK)

另外缺少什么: 组上的顺序最后计算。我希望显示最多人数的组(全名,natcode)

要求更进一步: 分组需要在另外两个字段上完成,因此就像4个字段一样。

如果我使用了错误的术语,请原谅。希望您对需要什么帮助有所了解。谢谢

1 个答案:

答案 0 :(得分:1)

以下是查询的方式。

必填查询(最终答案)

POST <your_index_name>/_search
{
  "size": 0,
  "query": {
    "bool": {
      "filter": {
        "term": {
          "birthDate": "18-sep-1993"
        }
      }
    }
  }, 
  "aggs": {
    "groupby_fullname": {
      "terms": {
        "field": "fullName.raw",
        "size": 2000
      },
      "aggs": {
        "natcode_filter": {
          "bucket_selector": {
            "buckets_path": {
              "hits": "groupby_natcode._bucket_count"
            },
            "script": "params.hits > 0"
          }
        },
        "groupby_natcode": {
          "terms": {
            "field": "natCode",
            "size": 2000,
            "min_doc_count": 2
          }
        }
      }
    }
  }
}

替代解决方案:(类似于选择不同的内容)

作为最后的选择,我能想到的是做一些基于fullName + "_" + natCode的select different。因此,基本上,您的密钥将采用JOHN SMITH_111的形式。确实会给您准确的结果,只是密钥将采用这种形式。

POST <your_index_name>/_search
{  
   "size":0,
   "query":{  
      "bool":{  
         "filter":{  
            "term":{  
               "birthDate":"18-sep-1993"
            }
         }
      }
   },
   "aggs":{  
      "name_count":{  
         "terms":{  
            "script":{  
               "inline":"doc['fullName.raw'].value + params.param + doc['natCode'].value",
               "lang":"painless",
               "params":{  
                  "param":"_"
               }
            }
         },
         "aggs":{  
            "my_filter":{  
               "bucket_selector":{  
                  "buckets_path":{  
                     "doc_count":"_count"
                  },
                  "script":"params.doc_count > 2"
               }
            }
         }
      }
   }
}

希望有帮助。