如何从elasticsearch连续删除旧的流利日志?

时间:2017-12-05 09:42:33

标签: elasticsearch logging fluentd elasticsearch-curator

Fluentd日志收集器写入Elasticsearch,最终填满磁盘。例如,如何将它们限制为一个月?

Fluentd配置的一部分(使用Kubernetes):

<match kubernetes.**>
  @type elasticsearch_dynamic
  host elasticsearch.default.svc.cluster.local
  port 9200
  include_tag_key true
  logstash_format true
  logstash_prefix kubernetes-${record['kubernetes']['pod_name']}
</match>

&#34;馆长&#34;对于Elasticsearch,可以删除&#34;索引&#34;,但我不知道Fluentd创建的索引,何时停止使用它们以及当仍有有用的新日志时删除索引意味着什么?

1 个答案:

答案 0 :(得分:1)

Curator将为您删除索引,无论Logstash,流利还是其他应用程序都是这样做的。此示例将与您在上述注释中提供的索引模式一起使用。

---
actions:
  1:
    action: delete_indices
    description: >-
      Delete indices older than 30 days (based on index name), for kubernetes-elasticsearch-
      prefixed indices. Ignore the error if the filter does not result in an
      actionable list of indices (ignore_empty_list) and exit cleanly.
    options:
      ignore_empty_list: true
      disable_action: true
    filters:
      - filtertype: pattern
        kind: prefix
        value: kubernetes-elasticsearch-
      - filtertype: age
        source: name
        direction: older
        timestring: '%Y.%m.%d'
        unit: days
        unit_count: 30