策展人重新索引操作-动态索引名称

时间:2018-10-15 14:44:30

标签: elasticsearch elasticsearch-5 logstash-configuration elasticsearch-curator

我们有整理脚本,每天晚上运行,以将每日索引重新索引为每月索引并删除每日索引

actions:
  1:
    description: "Reindex application-YYYY.MM.dd into application-YYYY.MM"
    action: reindex
    options:
      disable_action: False
      wait_interval: 9
      max_wait: -1
      request_body:
        source:
          index: REINDEX_SELECTION
        dest:
          index: application-2018.10 #change this
    filters:
    - filtertype: pattern
      kind: prefix
      value: application-
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: days
      unit_count: 1
  2:
    action: delete_indices
    description: >-
      Delete application-%Y.%m.%d indices older than 1 day
    options:
      ignore_empty_list: True
      disable_action: False
    filters:
    - filtertype: pattern
      kind: regex
      value: '^application-\d{4}\.\d{2}.\d{2}'
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: days
      unit_count: 1

我们如何动态制作每月索引名称?我想制作这样的应用程序-%Y.%m,而不是硬编码年月应用程序-2018.10

当我像下面那样使用application-%Y.%m时,它不起作用。

  request_body:
    source:
      index: REINDEX_SELECTION
    dest:
      index: application-%Y.%m

2 个答案:

答案 0 :(得分:0)

答案是在该表达式中使用date math。一个适合您的有效示例可能是:

dest:
  index: <application-{now/M{YYYY.MM}}>

更大的问题可能是,“如何确保该月的最后一天不会出现在下个月的索引中?”这种方法绝对会导致这种结果。如果您不在乎,那很好。如果您愿意,我们可以讨论可能的解决方案。

答案 1 :(得分:0)

这现在可能不再有用,但是我在寻找同一问题的答案时发现了这个问题,这就是我如何解决它(在untergeek的最初答案的帮助下):

actions:
  1:
    action: create_index
    description: "Create index: This is a monthly task"
    options:
      name: '<security-{now/d-1d{YYYY.MM}}>'
      continue_if_exception: True
  2:
    action: reindex
    description: "Reindex daily into monthly"
    options:
      wait_interval: 1
      max_wait: -1
      request_body:
        source:
          index: '<security-{now/d-1d{YYYY.MM.dd}}>'
          query:
            term:
              user: kimchy
        dest:
          index: '<security-{now/d-1d{YYYY.MM}}>'
    filters:
    - filtertype: none
  3:
    action: delete_indices
    description: "Delete yesterday's daily index"
    options:
      ignore_empty_list: True
    filters:
    - filtertype: pattern
      kind: regex
      value: '^security-\d{4}\.\d{2}.\d{2}'
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: days
      unit_count: 1
    - filtertype: age
      source: name
      direction: younger
      timestring: '%Y.%m.%d'
      unit: days
      unit_count: 2

我还在做一个查询过滤器,作为您可以忽略的重新索引的一部分,并且还将删除限制为恰好为1天的索引,因此,如果计划的重新索引失败,则后续的策展人作业将不会删除该索引