我在日志中的时间戳格式如下
2016-04-07 18:11:38.169 which is yyyy-MM-dd HH:mm:ss.SSS
这个日志文件不是实时文件(存储/旧文件),我试图用logstash @timestamp值替换这个timpestamp,以便在Kibana Visualization中获得更好的效果。
我在logstash中的过滤器如下所示
grok {
match => {
"message" => [ "(?<timestamp>(\d){4}-(\d){2}-(\d){2} (\d){2}:(\d){2}:(\d){2}.(\d){3}) %{SYSLOG5424SD} ERROR u%{BASE16FLOAT}.%{JAVACLASS} - TransId:2b948ed5-12c0-4ae0-9b99-f1ee01191001 - TransactionId ::\"2b948ed5-12c0-4ae0-9b99-f1ee01191001\"- Actual Time taken to process \:\: %{NUMBER:responseTime:int}" ]
}
}
date {
match => [ "timestamp:date" , "yyyy-MM-dd HH:mm:ss.SSS Z" ]
timezone => "UTC"
target => "@timestamp"
}
但是,它没有取代@timestamp值,Json值
{
"_index": "logstash-2017.02.09",
"_type": "logs",
"_id": "AVoiZq2ITxwgj2avgkZa",
"_score": null,
"_source": {
"path": "D:\\SoftsandTools\\Kibana\\Logs_ActualTimetakentoprocess.log",
"@timestamp": "2017-02-09T10:23:58.778Z", **logstash @timestamp**
"responseTime": 43,
"@version": "1",
"host": "4637",
"message": "2016-04-07 18:07:01.809 [SimpleAsyncTaskExecutor-3] ERROR s.v.wsclient.RestClient - TransId:2b948ed5-12c0-4ae0-9b99-f1ee01191001 - TransactionId ::\"2b948ed5-12c0-4ae0-9b99-f1ee01191001\"- Actual Time taken to process :: 43",
"timestamp": "2016-04-07 18:07:01.809" **Mine time stamp**
}
示例日志行 -
2016-04-07 18:11:38.171 [SimpleAsyncTaskExecutor-1] ERROR s.v.wsclient.RestClient - TransId:2b948ed5-12c0-4ae0-9b99-f1ee01191001 - TransactionId ::"2b948ed5-12c0-4ae0-9b99-f1ee01191001"- Actual Time taken to process :: 521
请你帮忙告诉我,我要在哪里拧...
答案 0 :(得分:3)
你应该基本上有一个 grok 匹配,以便使用你的日志行的timestamp
:
grok {
patterns_dir => ["give your path/patterns"]
match => { "message" => "^%{LOGTIMESTAMP:logtimestamp}%{GREEDYDATA}" }
}
在你的模式文件中,确保在日志中有与timestamp
匹配的模式,这可能如下所示:
LOGTIMESTAMP %{YEAR}%{MONTHNUM}%{MONTHDAY} %{TIME}
然后,当您完成grok
过滤后,您可以使用过滤后的值,如:
mutate {
add_field => { "newtimestamp" => "%{logtimestamp}" }
remove_field => ["logtimestamp"]
}
date {
match => [ "newtimestamp" , "ISO8601" , "yyyy-MM-dd HH:mm:ss.SSS" ]
target => "@timestamp" <-- the timestamp which you wanted to apply on
locale => "en"
timezone => "UTC"
}
希望这有帮助!
答案 1 :(得分:0)
您可以使用logstash的日期过滤器插件
date {
match => ["timestamp", "UNIX"]
}