弹性(搜索):获取具有最大和最小时间戳值的文档

时间:2015-03-20 15:01:29

标签: search elasticsearch

我遇到了搜索问题我无法弄清楚如何做到这一点。我的文档具有以下形式:

{
"timestamp":"2015-03-17T15:05:04.563Z",
"session_id":"1",
"user_id":"jan"
}

假设会话ID的第一个时间戳是“登录”,最后一个时间戳是“注销”。我希望所有会话都有“登录”和“注销”文档(如果可能,请按user_id排序)。我设法通过聚合获得正确的时间戳:

{
"aggs" : {
    "group_by_uid" : {
        "terms" : { 
            "field" : "user_id"
        },
        "aggs" : {
            "group_by_sid" : {
                "terms" : {
                    "field" : "session_id"
                },
                "aggs" : {
                    "max_date" : {
                        "max": { "field" : "timestamp" }
                    },
                    "min_date" : {
                        "min": { "field" : "timestamp" }
                    }
                }
            }
        }
    }
}
}

但是如何获得相应的文档呢?我也不介意我是否必须进行2次搜索(一次用于登录,一次用于注销)。我试过热门命中聚合和排序的东西,但我总是得到解析错误:/

我希望有人能给我一个提示:)

祝你好运, 扬

2 个答案:

答案 0 :(得分:17)

这是基于Sloan Ahrens提出的方法的单一搜索解决方案。优点是开始和结束会话条目位于同一个桶中。

{
"aggs": {
  "group_by_uid": {
     "terms": {
        "field": "user_id"
     },
     "aggs": {
        "group_by_sid": {
           "terms": {
              "field": "session_id"
           },
           "aggs": {
              "session_start": {
                 "top_hits": {
                    "size": 1,
                    "sort": [ { "timestamp": { "order": "asc" } } ]
                 }
              },
              "session_end": {
                 "top_hits": {
                    "size": 1,
                    "sort": [ { "timestamp": { "order": "desc" } } ]
                 }
              }
           }
        }
     }
  }
}
}

干杯, 扬

答案 1 :(得分:5)

你已经很亲密了。这个怎么样。使用两个搜索,每个搜索都会按照您的方式进行汇总,但也会在top_hit上获得第一个"timestamp"排序。

我只是设置了一个基本索引并添加了一些看起来像你发布的数据:

PUT /test_index
{
    "settings": {
        "number_of_shards": 1
    }
}

POST /test_index/_bulk
{"index":{"_index":"test_index","_type":"doc","_id":1}}
{"timestamp":"2015-03-17T15:05:04.563Z","session_id":"1","user_id":"jan"}
{"index":{"_index":"test_index","_type":"doc","_id":2}}
{"timestamp":"2015-03-17T15:10:04.563Z","session_id":"1","user_id":"jan"}
{"index":{"_index":"test_index","_type":"doc","_id":3}}
{"timestamp":"2015-03-17T15:15:04.563Z","session_id":"1","user_id":"jan"}
{"index":{"_index":"test_index","_type":"doc","_id":4}}
{"timestamp":"2015-03-17T18:05:04.563Z","session_id":"1","user_id":"bob"}
{"index":{"_index":"test_index","_type":"doc","_id":5}}
{"timestamp":"2015-03-17T18:10:04.563Z","session_id":"1","user_id":"bob"}
{"index":{"_index":"test_index","_type":"doc","_id":6}}
{"timestamp":"2015-03-17T18:15:04.563Z","session_id":"1","user_id":"bob"}

然后我可以通过以下方式获得每个会话的开始时间:

POST /test_index/_search?search_type=count
{
   "aggs": {
      "group_by_uid": {
         "terms": {
            "field": "user_id"
         },
         "aggs": {
            "group_by_sid": {
               "terms": {
                  "field": "session_id"
               },
               "aggs": {
                  "session_start": {
                     "top_hits": {
                        "size": 1,
                        "sort": [ { "timestamp": { "order": "asc" } } ]
                     }
                  }
               }
            }
         }
      }
   }
}
...
{
   "took": 5,
   "timed_out": false,
   "_shards": {
      "total": 1,
      "successful": 1,
      "failed": 0
   },
   "hits": {
      "total": 6,
      "max_score": 0,
      "hits": []
   },
   "aggregations": {
      "group_by_uid": {
         "buckets": [
            {
               "key": "bob",
               "doc_count": 3,
               "group_by_sid": {
                  "buckets": [
                     {
                        "key": "1",
                        "doc_count": 3,
                        "session_start": {
                           "hits": {
                              "total": 3,
                              "max_score": null,
                              "hits": [
                                 {
                                    "_index": "test_index",
                                    "_type": "doc",
                                    "_id": "4",
                                    "_score": null,
                                    "_source": {
                                       "timestamp": "2015-03-17T18:05:04.563Z",
                                       "session_id": "1",
                                       "user_id": "bob"
                                    },
                                    "sort": [
                                       1426615504563
                                    ]
                                 }
                              ]
                           }
                        }
                     }
                  ]
               }
            },
            {
               "key": "jan",
               "doc_count": 3,
               "group_by_sid": {
                  "buckets": [
                     {
                        "key": "1",
                        "doc_count": 3,
                        "session_start": {
                           "hits": {
                              "total": 3,
                              "max_score": null,
                              "hits": [
                                 {
                                    "_index": "test_index",
                                    "_type": "doc",
                                    "_id": "1",
                                    "_score": null,
                                    "_source": {
                                       "timestamp": "2015-03-17T15:05:04.563Z",
                                       "session_id": "1",
                                       "user_id": "jan"
                                    },
                                    "sort": [
                                       1426604704563
                                    ]
                                 }
                              ]
                           }
                        }
                     }
                  ]
               }
            }
         ]
      }
   }
}

和结束时间:

POST /test_index/_search?search_type=count
{
   "aggs": {
      "group_by_uid": {
         "terms": {
            "field": "user_id"
         },
         "aggs": {
            "group_by_sid": {
               "terms": {
                  "field": "session_id"
               },
               "aggs": {
                  "session_end": {
                     "top_hits": {
                        "size": 1,
                        "sort": [ { "timestamp": { "order": "desc" } } ]
                     }
                  }
               }
            }
         }
      }
   }
}
...
{
   "took": 2,
   "timed_out": false,
   "_shards": {
      "total": 1,
      "successful": 1,
      "failed": 0
   },
   "hits": {
      "total": 6,
      "max_score": 0,
      "hits": []
   },
   "aggregations": {
      "group_by_uid": {
         "buckets": [
            {
               "key": "bob",
               "doc_count": 3,
               "group_by_sid": {
                  "buckets": [
                     {
                        "key": "1",
                        "doc_count": 3,
                        "session_end": {
                           "hits": {
                              "total": 3,
                              "max_score": null,
                              "hits": [
                                 {
                                    "_index": "test_index",
                                    "_type": "doc",
                                    "_id": "6",
                                    "_score": null,
                                    "_source": {
                                       "timestamp": "2015-03-17T18:15:04.563Z",
                                       "session_id": "1",
                                       "user_id": "bob"
                                    },
                                    "sort": [
                                       1426616104563
                                    ]
                                 }
                              ]
                           }
                        }
                     }
                  ]
               }
            },
            {
               "key": "jan",
               "doc_count": 3,
               "group_by_sid": {
                  "buckets": [
                     {
                        "key": "1",
                        "doc_count": 3,
                        "session_end": {
                           "hits": {
                              "total": 3,
                              "max_score": null,
                              "hits": [
                                 {
                                    "_index": "test_index",
                                    "_type": "doc",
                                    "_id": "3",
                                    "_score": null,
                                    "_source": {
                                       "timestamp": "2015-03-17T15:15:04.563Z",
                                       "session_id": "1",
                                       "user_id": "jan"
                                    },
                                    "sort": [
                                       1426605304563
                                    ]
                                 }
                              ]
                           }
                        }
                     }
                  ]
               }
            }
         ]
      }
   }
}

这是我使用的代码:

http://sense.qbox.io/gist/05edb48b840e6a992646643913db8ef0a3ccccb3