Question

以下是我的日志示例：

23:28:32.226 WARN  [MsgParser:ListProc-Q0:I5]   Parsing error
Error mapping the fieldAdditional Information: 

    at com.authentic.mapper.parsing.LengthVar.readBytes(LengthVar.java:178)
    at com.authentic.mapper.parsing.GrpLengthVar.read(GrpLengthVar.java:96)
    at com.authentic.mapper.parsing.GrpLengthVar.read(GrpLengthVar.java:119)
    at com.authentic.mapper.parsing.MsgParser.processReadEnumeration(MsgParser.java:339)
    at com.authentic.mapper.parsing.MsgParser.parseIncomingMessageBody(MsgParser.java:295)
    at com.authentic.mapper.MapperMgr.parseMsg(MapperMgr.java:1033)
    at com.authentic.architecture.interchange.accesspoint.AbstractConnectionHandler.parseMessage(AbstractConnectionHandler.java:4408)
    at com.authentic.architecture.interchange.accesspoint.AbstractConnectionHandler.plainMessageReceivedEvent(AbstractConnectionHandler.java:2031)
    at com.authentic.architecture.interchange.accesspoint.AbstractConnectionHandler.messageReceivedEvent(AbstractConnectionHandler.java:1911)
    at com.authentic.architecture.interchange.accesspoint.SocketConnectionHandler.messageReceivedEvent(SocketConnectionHandler.java:801)
    at com.authentic.architecture.interchange.accesspoint.SocketConnectionHandler.messageReceivedEvent(SocketConnectionHandler.java:282)
    at com.authentic.architecture.interchange.accesspoint.SocketConnectionHandler.messageReceivedEvent(SocketConnectionHandler.java:261)
    at com.authentic.architecture.interchange.accesspoint.AbstractConnectionHandler.processEventQueue(AbstractConnectionHandler.java:4110)
    at com.authentic.architecture.interchange.accesspoint.AbstractConnectionHandler.access$100(AbstractConnectionHandler.java:320)
    at com.authentic.architecture.interchange.accesspoint.AbstractConnectionHandler$ConnectionHandlerRunner.execute(AbstractConnectionHandler.java:416)
    at com.authentic.architecture.actions.ListProcessor.suspend(ListProcessor.java:1130)
    at com.authentic.architecture.actions.ListProcessor.run(ListProcessor.java:775)
    at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.NumberFormatException: For input string: "^123"
    at java.lang.NumberFormatException.forInputString(Unknown Source)
    at java.lang.Integer.parseInt(Unknown Source)
    at java.lang.Integer.parseInt(Unknown Source)
    at com.authentic.mapper.parsing.LengthVar.readBytes(LengthVar.java:170)
    ... 17 more

我必须将这些日志解析为以下字段：timestamp，log-level，logger，msg，stacktrace。

我使用过多线过滤器：

multiline {
pattern => "%{TIME:timestamp}"
negate => true
what => “previous”
}

和我在grok过滤器中使用的模式：

match=>{"message"=>"%{TIME:timestamp} %{LOGLEVEL:loglevel} \s*\[%{DATA:logger}\]\s*%{GREEDYDATA:msg}\n*(?<stacktrace>(.|\r|\n)*)"}

我已使用http://grokconstructor.appspot.com/do/match进行了检查。但是堆栈跟踪字段得到了这个匹配错误。

请提出一些建议。提前谢谢。

Answer 1

如果要匹配整个堆栈跟踪，则需要多行过滤器。这个多行过滤器应该适合您：

codec => multiline {
        pattern => "^%{TIME} "
        negate => true
        what => previous
    }

说明：每个不以时间戳开头的行（如23：28：32.226）将作为上一行的一部分进行重新定位。另请参阅有关处理多行的docs。

现在你的模式。以下是我的工作：

%{TIME:timestamp} %{LOGLEVEL:loglevel}  \[%{DATA:logger}\]   %{GREEDYDATA:message}\n(?<stacktrace>(.|\r|\n)*)

非常自我解释，我希望：使用\[和\]，\n转义大括号[和]，以匹配换行符。另请注意条目之间的空格。

对于最后一部分（stacktrace），另请参阅this question有关如何匹配包括换行符在内的所有内容。

完整配置可能如下所示：

input {
  file {
    path => "/var/log/yourlog.log"
    start_position => "beginning"
    codec => multiline {
        pattern => "^%{TIME} "
        negate => true
        what => previous
    }
  }
}
filter {
  grok {
    match => [ "message", "%{TIME:timestamp} %{LOGLEVEL:loglevel}  \[%{DATA:logger}\]   %{GREEDYDATA:message}\n(?<stacktrace>(.|\r|\n)*)" ]
  }
}

http://grokconstructor.appspot.com的结果：

用于日志的自定义Grok模式

1 个答案: