通过logtash中的grok过滤器在给定的日志结构上解析日志文本

时间:2019-04-07 08:29:53

标签: elasticsearch logstash

2019-04-01 10:57:35|[a1vx4d9r - ecaf-myself - N2PENL-ECFA0141.india.airtel.itm - servlet:/reserveNumbers/getAvailableNumbers/v1?httpMethodRestrict=POST ][[ACTIVE] ExecuteThread: '36' for queue: 'weblogic.kernel.Default (self-tuning)'] INFO|com.airtel.common.aop.LogExecutionTime|EXECUTION TIME LOGGING METHODNAME: postResponseFromESB EXECUTION TIME: 472 ms

上面是日志格式。

请帮助在logstash中编写grok过滤器,以分隔字段,如下所示:

timestamp: 2019-04-01 10:57:35
user_id:a1vx4d9r
project name: ecaf-myself
host_name: N2PENL
api_name:ECFA0141.india.airtel.itm - servlet:/reserveNumbers/getAvailableNumbers/v1?httpMethodRestrict=POST 
thread_id: [ACTIVE] ExecuteThread: '36' for queue: 'weblogic.kernel.Default (self-tuning)'
log_level: INFO
Method_name:postResponseFromESB 
Method_time:472

1 个答案:

答案 0 :(得分:2)

使用已经定义的所有here所需模式,编写grok过滤器很容易。您可以使用此应用来尝试使用grok过滤器,并查看它是否与您的日志模式匹配:http://grokdebug.herokuapp.com/

对于上面的示例,下面的grok过滤器将起作用。您可以使用上面的两个资源来对其进行检查和微调,以适应您的需求:

%{TIMESTAMP_ISO8601}.*\[%{WORD:user_id}* - %{NOTSPACE:project_name}* - %{WORD:host_name}*-%{DATA:api_name} \]%{GREEDYDATA:thread}\] %{WORD:log_level}.*METHODNAME: %{WORD:Method_name}.*EXECUTION TIME: %{INT:Method_time}