Question

日志：

20160927-210452.110|I|cpeg-001.anc.com|test_app-1.5-0||~|f324dfsdf23sd23||org.springframework.orm.hibernate3.LocalSessionFactoryBean:777|Building new Hibernate SessionFactory

过滤

filter {
  grok{
     match => [ "message", "(?<date_time>[0-9.-]+)(?<delimiter>[|])%{WORD:method}(?<delimiter>[|])%{USERNAME:host_name}(?<delimiter>[|])%{USERNAME:app_name_version}(?<delimiter>[|~]+)%{USERNAME:session}(?<delimiter>[|]+)(?<class_name>.+)(?<delimiter>[|])(?<log_message>.+)" ]

     remove_field => [ "delimiter" ]
   }
}

有没有办法跳过＆＃39; |＆＃39;除了＆＃39; remove_field＆＃39; ？

Answer 1

为什么不在你的grok模式中硬编码|？您可以使用\|

将其转义

<强>模式：

 (?<date_time>[0-9.-]+)\|%{WORD:method}\|%{USERNAME:host_name}\|%{USERNAME:app_name_version}\|\|\~\|%{USERNAME:session}\|\|%{DATA:classname}\|%{GREEDYDATA:logmessage}

过滤

grok{ match => [ "message", "(?<date_time>[0-9.-]+)\|%{WORD:method}\|%{USERNAME:host_name}\|%{USERNAME:app_name_version}\|\|\~\|%{USERNAME:session}\|\|%{DATA:classname}\|%{GREEDYDATA:logmessage}" ] }

有效。您可以使用grok debugger进行测试。

Answer 2

谢谢，我刚刚在./custom_patterns/my_pattern中创建了自定义模式：

CUST_DATETIME [0-9.-]+
SEPARATOR \|
MULTI_SEPARATOR \|\|\~\|

在过滤器匹配中更新了相同内容：

filter {
  grok{
    patterns_dir => "./custom_patterns"

    match => [ "message", "%{CUST_DATETIME:orb_date}%{SEPARATOR}%{WORD:method}%{SEPARATOR}%{USERNAME:host_name}%{SEPARATOR}%{USERNAME:app_name_version}%{MULTI_SEPARATOR}%{USERNAME:session}%{SEPARATOR}%{DATA:class_name}%{SEPARATOR}%{GREEDYDATA:log_message}" ]

   }
}

logstash的自定义消息过滤器

2 个答案: