Question

我有两种类型的错误消息，格式如下：

[2017-05-25 01:00:00,647][ERROR][marvel.agent.exporter.local] local exporter [default_local] - failed to delete indices
RemoteTransportException[[data-0][10.0.0.8:9300][indices:admin/delete]]; nested: IndexNotFoundException[no such index];

[2017-05-18 00:00:06,339][DEBUG][action.admin.indices.create] [data-2] [data-may-2017,data-apr-2017,data-mar-2017] failed to create
[data-may-2017,data-apr-2017,data-mar-2017]

我的logstash配置如下：

input {
      file {
            path => "D:\logstash\logstash-2.4.0\bin\logs.txt"
            start_position => "beginning"
        codec => multiline {
            pattern => "^\[%{TIMESTAMP_ISO8601:TIMESTAMP}\]"
            negate => true
            what => "previous"
        }
  }

}
filter {
   grok {
        match => [ "message", "(?m)^\[%{TIMESTAMP_ISO8601:TIMESTAMP}\]\[%{LOGLEVEL:LEVEL}%{SPACE}\]\[%{DATA:ERRORTYPE}\]%{SPACE}\[%{DATA:SERVERNAME}\]%{SPACE}(?<ERRORMESSAGE>(.|\r|\n)*)", "message",  "(?m)^\[%{TIMESTAMP_ISO8601:TIMESTAMP}\]\[%{LOGLEVEL:LEVEL}%{SPACE}\]\[%{DATA:ERRORTYPE}%{SPACE}\]%{SPACE}(?<ERRORMESSAGE>(.|\r|\n)*)"]
   }

}
output {

  stdout { codec => rubydebug }
}

对于两个日志，它只采用第一个grok模式。为什么不采取第二个？

Answer 1

似乎我的第一个grok模式匹配所有日志，所以这就是为什么logstash只采用第一个模式。所以我使用了下面的配置if if条件工作正常。

input {
      file {
            path => "D:\logstash\logstash-2.4.0\bin\logs.txt"
            start_position => "beginning"
             type => "log"
        codec => multiline {
            pattern => "^\[%{TIMESTAMP_ISO8601:TIMESTAMP}\]"
            negate => true
            what => "previous"
        }
  }

}
    filter {
      if [type] == "log" {
        grok {
          match => [ "message", "(?m)^\[%{TIMESTAMP_ISO8601:TIMESTAMP}\]\[%{LOGLEVEL:LEVEL}%{SPACE}\]\[%{DATA:ERRORTYPE}%{SPACE}\]%{SPACE}(?<ERRORMESSAGE>(.|\r|\n)*)"]
        }
    # DEBUG Logs
    if "grokked" not in [tags] and "DEBUG" == [LEVEL] {
    grok { match => [ "ERRORMESSAGE", "(?m)^\[%{DATA:SERVERNAME}\]" ]
    add_tag => [ "Debug Logs", "grokked" ]
    tag_on_failure => [ ]
    }
    }
    }
    }
output {

  stdout { codec => rubydebug }
}

Answer 2

你的问题是：

为什么不采取第二个？

答案在这里：

filter {
   grok {
        match => [ "message", "(?m)^\[%{TIMESTAMP_ISO8601:TIMESTAMP}\]\[%{LOGLEVEL:LEVEL}%{SPACE}\]\[%{DATA:ERRORTYPE}\]%{SPACE}\[%{DATA:SERVERNAME}\]%{SPACE}(?<ERRORMESSAGE>(.|\r|\n)*)", "message",  "(?m)^\[%{TIMESTAMP_ISO8601:TIMESTAMP}\]\[%{LOGLEVEL:LEVEL}%{SPACE}\]\[%{DATA:ERRORTYPE}%{SPACE}\]%{SPACE}(?<ERRORMESSAGE>(.|\r|\n)*)"]
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------^ 
   }

}

您不必多次指定源，只需一次。

你现在做的是：

["message", "pattern", "message", "pattern"]

实际上它必须是：

["message", "pattern", "pattern", ..., "pattern"]

在Logstash中匹配多个模式？

2 个答案: