Question

我想在ELK stack 6.3.2版本中解析一个标准的JAVA异常，如下所示：

2018-09-04 05:29:03.955 [default task-38] ERROR c.r.e.u.util.MongoConnectionUtil.createMongoUser - Exception occured while creating mongo userCommand failed with error 11000: 'User "asdf" already exists' on server 192.168.1.33:27017. The full response is { "ok" : 0.0, "errmsg" : "User \"asdf\" already exists", "code" : 11000, "codeName" : "DuplicateKey" }
com.mongodb.MongoCommandException: Command failed with error 11000: 'User "qwer" already exists' on server 192.168.1.33:27017. The full response is { "ok" : 0.0, "errmsg" : "User \"asdf\" already exists", "code" : 11000, "codeName" : "DuplicateKey" }
    at com.mongodb.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:115)
    at com.mongodb.connection.CommandProtocol.execute(CommandProtocol.java:114) ...

我的filebeat.yml具有配置：

filebeat.inputs:

- type: log
  enabled: true
  paths:
    - C:\logs\test.log
  multiline.pattern: '^[[:space:]]+(at|\.{3})\b|^Caused by:'
  multiline.negate: false
  multiline.match: after

和我的logstash.conf输入类似：

input {

beats {
    port=>5044
        codec => multiline {
               pattern => "^\s"
              what => "previous"
}
}

但是logstash说无法解析模式，实际上它崩溃并带有异常。如果仅删除codec配置，则将解析异常的第一行。我也在https://discuss.elastic.co/t/multiline-parsing-patterns/147171提出了同样的问题，但没有任何回应。

Answer 1

您需要更改： multiline.negate至true。而且我不确定您要使用该模式实现什么，但是看来您应该使用：

multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'

此外，您无需在logstash中使用多行-只需简单地进行以下操作即可：

input {
    beats {
        port => 5044            
    }
}

在夏季，为了捕获所有日志，我将您的Filebeat配置更改为：

- type: log
  enabled: true
  paths:
    - C:\logs\test.log
  multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
  multiline.negate: true
  multiline.match: after

一些解释：当我们选择negate:true和match: after时，我们告诉FileBeat：

与模式不匹配的连续行被附加到前一行确实匹配。

换句话说-它告诉FileBeat收获以给定模式开始的每一行，并在该模式再次出现在新行的开头时停止。对于此模式^[0-9]{4}-[0-9]{2}-[0-9]{2}，如果您遇到以下2个例外情况：

2018-09-04 05:29:03.955 [default task-38] ERROR c.r.e.u.util.MongoConnectionUtil.createMongoUser - Exception occured while creating mongo userCommand failed with error 11000: 'User "asdf" already exists' on server 192.168.1.33:27017. The full response is { "ok" : 0.0, "errmsg" : "User \"asdf\" already exists", "code" : 11000, "codeName" : "DuplicateKey" }
com.mongodb.MongoCommandException: Command failed with error 11000: 'User "qwer" already exists' on server 192.168.1.33:27017. The full response is { "ok" : 0.0, "errmsg" : "User \"asdf\" already exists", "code" : 11000, "codeName" : "DuplicateKey" }
    at com.mongodb.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:115)
    at com.mongodb.connection.CommandProtocol.execute(CommandProtocol.java:114) ...

2018-09-04 05:30:00.000 [default task-38] ERROR c.r.e.u.util.MongoConnectionUtil.createMongoUser - Exception occured while creating mongo userCommand failed with error 11000: 'User "asdf" already exists' on server 192.168.1.33:27017. The full response is { "ok" : 0.0, "errmsg" : "User \"asdf\" already exists", "code" : 11000, "codeName" : "DuplicateKey" }
com.mongodb.MongoCommandException: Command failed with error 11000: 'User "qwer" already exists' on server 192.168.1.33:27017. The full response is { "ok" : 0.0, "errmsg" : "User \"asdf\" already exists", "code" : 11000, "codeName" : "DuplicateKey" }
    at com.mongodb.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:115)
    at com.mongodb.connection.CommandProtocol.execute(CommandProtocol.java:114) ...

它将捕获每个异常作为不同的条目日志。如果您记录了更多内容，并且希望文件信号仅收获错误，那就不一样了。在我们的程序中，我们将全部收集并按严重性（即错误，信息，警告等）进行查询

多行解析模式

1 个答案: