我需要使用ES中的3个字段进行聚合(分组)。
我可以在1个查询中执行此操作,还是需要为每列使用facet + iterate?
谢谢
答案 0 :(得分:28)
从ElasticSearch
版本1.0开始,新aggregations API允许使用子聚合按多个字段进行分组。假设您要按字段field1
,field2
和field3
进行分组:
{
"aggs": {
"agg1": {
"terms": {
"field": "field1"
},
"aggs": {
"agg2": {
"terms": {
"field": "field2"
},
"aggs": {
"agg3": {
"terms": {
"field": "field3"
}
}
}
}
}
}
}
}
当然,这可以用于你喜欢的任何领域。
<强>更新强>
为了完整性,以下是上述查询的输出的外观。下面是用于生成聚合查询并将结果展平为字典列表的python代码。
{
"aggregations": {
"agg1": {
"buckets": [{
"doc_count": <count>,
"key": <value of field1>,
"agg2": {
"buckets": [{
"doc_count": <count>,
"key": <value of field2>,
"agg3": {
"buckets": [{
"doc_count": <count>,
"key": <value of field3>
},
{
"doc_count": <count>,
"key": <value of field3>
}, ...
]
},
{
"doc_count": <count>,
"key": <value of field2>,
"agg3": {
"buckets": [{
"doc_count": <count>,
"key": <value of field3>
},
{
"doc_count": <count>,
"key": <value of field3>
}, ...
]
}, ...
]
},
{
"doc_count": <count>,
"key": <value of field1>,
"agg2": {
"buckets": [{
"doc_count": <count>,
"key": <value of field2>,
"agg3": {
"buckets": [{
"doc_count": <count>,
"key": <value of field3>
},
{
"doc_count": <count>,
"key": <value of field3>
}, ...
]
},
{
"doc_count": <count>,
"key": <value of field2>,
"agg3": {
"buckets": [{
"doc_count": <count>,
"key": <value of field3>
},
{
"doc_count": <count>,
"key": <value of field3>
}, ...
]
}, ...
]
}, ...
]
}
}
}
以下python代码在给定字段列表的情况下执行group-by。我指定include_missing=True
,它还包含缺少某些字段的值组合(如果您拥有this的Elasticsearch 2.0版,则不需要它)
def group_by(es, fields, include_missing):
current_level_terms = {'terms': {'field': fields[0]}}
agg_spec = {fields[0]: current_level_terms}
if include_missing:
current_level_missing = {'missing': {'field': fields[0]}}
agg_spec[fields[0] + '_missing'] = current_level_missing
for field in fields[1:]:
next_level_terms = {'terms': {'field': field}}
current_level_terms['aggs'] = {
field: next_level_terms,
}
if include_missing:
next_level_missing = {'missing': {'field': field}}
current_level_terms['aggs'][field + '_missing'] = next_level_missing
current_level_missing['aggs'] = {
field: next_level_terms,
field + '_missing': next_level_missing,
}
current_level_missing = next_level_missing
current_level_terms = next_level_terms
agg_result = es.search(body={'aggs': agg_spec})['aggregations']
return get_docs_from_agg_result(agg_result, fields, include_missing)
def get_docs_from_agg_result(agg_result, fields, include_missing):
current_field = fields[0]
buckets = agg_result[current_field]['buckets']
if include_missing:
buckets.append(agg_result[(current_field + '_missing')])
if len(fields) == 1:
return [
{
current_field: bucket.get('key'),
'doc_count': bucket['doc_count'],
}
for bucket in buckets if bucket['doc_count'] > 0
]
result = []
for bucket in buckets:
records = get_docs_from_agg_result(bucket, fields[1:], include_missing)
value = bucket.get('key')
for record in records:
record[current_field] = value
result.extend(records)
return result
答案 1 :(得分:7)
您可以通过两种方式实现:
1)在单个方面结果中使用多个字段:
单个字段facet的示例:
curl -X GET "http://localhost:9200/sales/order/_search?pretty=true" -d '{
"query": {
"query_string": {
"query": "shohi*",
"fields": [
"billing_name"
]
}
},
"facets": {
"facet_result": {
"terms": {
"fields": [
"status"
],
"order": "term",
"size": 15
}
}
}
}'
单面结果中多个字段的示例:
curl -X GET "http://localhost:9200/sales/order/_search?pretty=true" -d '{
"query": {
"query_string": {
"query": "shohi*",
"fields": [
"billing_name"
]
}
},
"facets": {
"facet_result": {
"terms": {
"fields": [
"status",
"customer_gender",
"state"
],
"order": "term",
"size": 15
}
}
}
}'
2)使用多个方面结果集:
curl -X GET "http://localhost:9200/sales/order/_search?pretty=true" -d '{
"query": {
"query_string": {
"query": "*",
"fields": [
"increment_id"
]
}
},
"facets": {
"status_facets": {
"terms": {
"fields": [
"status"
],
"size": 50,
"order": "term"
}
},
"gender_facets": {
"terms": {
"fields": [
"customer_gender"
]
}
},
"state_facets": {
"terms": {
"fields": [
"state"
],
,
"order": "term"
}
}
}
}'
参考链接: http://www.elasticsearch.org/guide/reference/api/search/facets/terms-facet.html