Question

使用消息的时间戳替换事件的时间戳对我不起作用。以下是详细信息。

示例日志：

20170119 144002.184140 0005B9427CA0_CU_1 user.notice ProcMon:: 10.220.0.13 is valid IP Address

模式：

CSSYSLOGTIMESTAMP  %{YEAR}%{MONTHNUM}%{MONTHDAY}[T ]%{HOUR}%{MINUTE}%{SECOND}.%{NONNEGINT:MSEC}

过滤器：

filter {
  grok {
    patterns_dir => ["/root/logstash-5.5.0/patterns"]
    match => { "message" => "^%{CSSYSLOGTIMESTAMP:syslog_timestamp} %{DATA:syslog_hostname} %{DATA:syslog_level} %{DATA:app_name}: %{GREEDYDATA:syslog_message}" }
  }

  date {
    match => ["syslog_timestamp" , "yyyyMMdd HHmmss.SSS"]
    target => "@timestamp"
    add_field => { "debug" => "timestampMatched"}
  }
}

Logstash Debug输出：

[2017-07-18T10:25:01,152][DEBUG][logstash.pipeline        ] filter received {"event"=>{"@timestamp"=>2017-07-18T04:54:55.170Z, "offset"=>747452, "@version"=>"1", "input_type"=>"log", "beat"=>{"hostname"=>"node1", "name"=>"node1", "version"=>"5.5.0"}, "host"=>"node1", "source"=>"/root/samplelogs/Debug.log", "message"=>"20170119 144002.184140 0005B9427CA0_CU_1 user.notice ProcMon:: 10.220.0.13 is valid IP Address", "type"=>"log", "tags"=>["beats_input_codec_plain_applied"]}}
[2017-07-18T10:25:01,154][DEBUG][logstash.filters.grok    ] Running grok filter {:event=>2017-07-18T04:54:55.170Z node1 20170119 144002.184140 0005B9427CA0_CU_1 user.notice ProcMon:: 10.220.0.13 is valid IP Address}
[2017-07-18T10:25:01,159][DEBUG][logstash.filters.grok    ] Event now:  {:event=>2017-07-18T04:54:55.170Z node1 20170119 144002.184140 0005B9427CA0_CU_1 user.notice ProcMon:: 10.220.0.13 is valid IP Address}
[2017-07-18T10:25:01,165][DEBUG][logstash.pipeline        ] output received {"event"=>{"MSEC"=>"184140", "offset"=>747452, "input_type"=>"log", "source"=>"/root/samplelogs/Debug.log", "message"=>"20170119 144002.184140 0005B9427CA0_CU_1 user.notice ProcMon:: 10.220.0.13 is valid IP Address", "type"=>"log", "syslog_message"=>"10.220.0.13 is valid IP Address", "tags"=>["beats_input_codec_plain_applied", "_dateparsefailure"], "app_name"=>"ProcMon:", "@timestamp"=>2017-07-18T04:54:55.170Z, "syslog_hostname"=>"0005B9427CA0_CU_1", "syslog_timestamp"=>"20170119 144002.184140", "@version"=>"1", "beat"=>{"hostname"=>"node1", "name"=>"node1", "version"=>"5.5.0"}, "host"=>"node1", "syslog_level"=>"user.notice"}}

我可以看到所有字段都被正确提取。消息时间戳在字段syslog_timestamp中创建。但是@timestamp没有被消息时间戳替换。我究竟做错了什么？感谢。

Answer 1

发现了这个问题。由于参考文档中提到的这个文本，我只使用了最多3位小数（SSS）。

"S: fraction of a second Maximum precision is milliseconds (SSS). Beyond that, zeroes are appended."

但事实证明，在定义过滤器时你仍然需要使用6个S。

Logstash：替换消息无效的时间戳

1 个答案: