配置文件拍以控制读取日志的频率并忽略旧日志

时间:2019-04-25 01:05:38

标签: elastic-stack filebeat

我的filebeat.yml中有一个探矿者设置,大致如下:

filebeat.propectors:
- type: log
paths:
  - /tmp/log/typeA*.log
pipeline: "pipelineA"
fields_under_root: true
fields:
  logtype: TYPEA
- type: log
paths:
  - /tmp/log/typeB*.log
pipeline: "pipelineB"
fields_under_root: true
fields:
  logtype: TYPEB
- type: log
paths:
  - /tmp/log/typeC*.log
pipeline: "pipelineC"
fields_under_root: true
fields:
  logtype: TYPEC

它工作正常,但我需要控制读取日志。我希望能够控制输入,以使某些日志的读取频率比(例如,对于pipelineC,每天仅读取一次,但是对于pipelineA,则每分钟读取一次)。我还希望能够忽略超过特定期限的日志(例如,忽略超过3天的日志)。

对于任何想法,我将不胜感激。谢谢

1 个答案:

答案 0 :(得分:0)

解决了这个问题。这是解决方案:

filebeat.propectors

type: log
paths:
/tmp/log/typeA*.log
ignore_older: 72h
scan_frequency: 10s
pipeline: "pipelineA"
fields_under_root: true
fields:
logtype: TYPEA
type: log
paths:
/tmp/log/typeB*.log
ignore_older: 72h
scan_frequency: 10s
pipeline: "pipelineB"
fields_under_root: true
fields:
logtype: TYPEB
type: log
paths:
/tmp/log/typeC*.log
ignore_older: 72h
scan_frequency: 10s
pipeline: "pipelineC"
fields_under_root: true
fields:
logtype: TYPEC