如何在Elasticsearch中执行嵌套日期和未嵌套日期之间的日期算术运算?

时间:2017-06-19 14:31:23

标签: date datetime elasticsearch elasticsearch-5 elasticsearch-painless

考虑以下Elasticsearch(v5.4)对象("奖励" doc类型):

{
  "name": "Gold 1000",
  "date": "2017-06-01T16:43:00.000+00:00",
  "recipient": {
    "name": "James Conroy",
    "date_of_birth": "1991-05-30"
  }
}

award.dateaward.recipient.date_of_birth的映射类型为" date"。

我想执行一个range aggregation来获取此奖项的收件人年龄范围列表(" 18岁以下"," 18-24",& #34; 24-30"," 30 +"),在获奖时。我尝试了以下聚合查询:

{
  "size": 0,
  "query": {"match_all": {}},
  "aggs": {
    "recipients": {
      "nested": {
        "path": "recipient"
      },
      "aggs": {
        "age_ranges": {
          "range": {
            "script": {
              "inline": "doc['date'].date - doc['recipient.date_of_birth'].date"
            },
            "keyed": true,
            "ranges": [{
              "key": "Under 18",
              "from": 0,
              "to": 18
            }, {
              "key": "18-24",
              "from": 18,
              "to": 24
            }, {
              "key": "24-30",
              "from": 24,
              "to": 30
            }, {
              "key": "30+",
              "from": 30,
              "to": 100
            }]
          }
        }
      }
    }
  }
}

问题1

但由于script部分的日期比较,我收到以下错误:

Cannot apply [-] operation to types [org.joda.time.DateTime] and [org.joda.time.MutableDateTime].

DateTime对象是award.date字段,MutableDateTime对象是award.recipient.date_of_birth字段。我尝试过像doc['recipient.date_of_birth'].date.toDateTime()这样的事情(尽管Joda文档声称MutableDateTime从父类继承了此方法,但它仍无效)。我也尝试过这样做的事情:

"script": "ChronoUnit.YEARS.between(doc['date'].date, doc['recipient.date_of_birth'].date)"

遗憾的是,这也不起作用:(

问题2

我注意到我这样做了:

"aggs": {
  "recipients": {
    "nested": {
      "path": "recipient"
    },
    "aggs": {
      "award_years": {
        "terms": {
          "script": {
            "inline": "doc['date'].date.year"
          }
        }
      }
    }
  }
}

我的1970 doc_count恰好等于ES中的文档总数。这让我相信,访问嵌套对象之外的属性根本不起作用,并给了我一些默认值,如epoch datetime。如果我做相反的事情(汇总出生日期没有筑巢),我会在所有出生日期(1970年,纪元日期时间)得到完全相同的东西。那么如何比较这两个日期呢?

我在这里绞尽脑汁,我觉得有一些聪明的解决方案超出了我目前对Elasticsearch的专业知识。救命啊!

如果你想设置一个快速的环境来帮助我,这里有一些卷曲的好处:

curl -XDELETE http://localhost:9200/joelinux
curl -XPUT http://localhost:9200/joelinux -d "{\"mappings\": {\"award\": {\"properties\": {\"name\": {\"type\": \"string\"}, \"date\": {\"type\": \"date\", \"format\": \"yyyy-MM-dd'T'HH:mm:ss.SSSSSSZ\"}, \"recipient\": {\"type\": \"nested\", \"properties\": {\"name\": {\"type\": \"string\"}, \"date_of_birth\": {\"type\": \"date\", \"format\": \"yyyy-MM-dd\"}}}}}}}"
curl -XPUT http://localhost:9200/joelinux/award/1 -d '{"name": "Gold 1000", "date": "2016-06-01T16:43:00.000000+00:00", "recipient": {"name": "James Conroy", "date_of_birth": "1991-05-30"}}'
curl -XPUT http://localhost:9200/joelinux/award/2 -d '{"name": "Gold 1000", "date": "2017-02-28T13:36:00.000000+00:00", "recipient": {"name": "Martin McNealy", "date_of_birth": "1983-01-20"}}'

那应该给你一个" joelinux"索引有两个"奖励"文档来测试这个(" James Conroy"和#34; Martin McNealy")。提前谢谢!

1 个答案:

答案 0 :(得分:3)

不幸的是,您无法在同一上下文中访问嵌套和非嵌套字段。作为一种变通方法,您可以使用copy_to选项将映射更改为自动将日期从嵌套文档复制到根上下文:

{
    "mappings": {
        "award": {
            "properties": {
                "name": {
                    "fields": {
                        "keyword": {
                            "ignore_above": 256,
                            "type": "keyword"
                        }
                    },
                    "type": "text"
                },
                "date": {
                    "type": "date"
                },
                "date_of_birth": {
                    "type": "date" // will be automatically filled when indexing documents
                },
                "recipient": {
                    "properties": {
                        "name": {
                            "fields": {
                                "keyword": {
                                    "ignore_above": 256,
                                    "type": "keyword"
                                }
                            },
                            "type": "text"
                        },
                        "date_of_birth": {
                            "type": "date",
                            "copy_to": "date_of_birth" // copy value to root document
                        }
                    },
                    "type": "nested"
                }
            }
        }
    }
}

之后,您可以使用路径date访问出生日期,但计算得到日期之间的年数有点棘手:

Period.between(LocalDate.ofEpochDay(doc['date_of_birth'].date.getMillis() / 86400000L), LocalDate.ofEpochDay(doc['date'].date.getMillis() / 86400000L)).getYears()

这里我将原始的JodaTime日期对象转换为system.time.LocalDate个对象:

  1. 从1970-01-01获取毫秒数
  2. 转换为1970-01-01的天数除以86400000L(一天的毫秒数)
  3. 转换为LocalDate对象
  4. 从两个日期创建基于日期的Period对象
  5. 获取两个日期之间的年数。
  6. 因此,最终的聚合查询如下所示:

    {
        "size": 0,
        "query": {
            "match_all": {}
        },
        "aggs": {
            "age_ranges": {
                "range": {
                    "script": {
                        "inline": "Period.between(LocalDate.ofEpochDay(doc['date_of_birth'].date.getMillis() / 86400000L), LocalDate.ofEpochDay(doc['date'].date.getMillis() / 86400000L)).getYears()"
                    },
                    "keyed": true,
                    "ranges": [
                        {
                            "key": "Under 18",
                            "from": 0,
                            "to": 18
                        },
                        {
                            "key": "18-24",
                            "from": 18,
                            "to": 24
                        },
                        {
                            "key": "24-30",
                            "from": 24,
                            "to": 30
                        },
                        {
                            "key": "30+",
                            "from": 30,
                            "to": 100
                        }
                    ]
                }
            }
        }
    }