Question

我有以下日志消息

2021-03-26 11:49:25.575: 2021-03-26 11:49:25.575 [INFO] 10.0.3.12 - "POST https://api.kr-seo.assistant.watson.cloud.ibm.com/instances/a33da834-a7a7-48c2-9bf6-d3207849ad71/v1/workspaces/c6e3035b-411a-468d-adac-1ae608f7bf68/message?version=2018-07-10" 200 462 ms

2021-03-26 11:49:26.514: 2021-03-26 11:49:26.514 [INFO] 10.0.3.12 + "POST http://test-bff.lotteon.com/order/v1/mylotte/getOrderList"

我想使用 logstash 之类的转换

"timestamp" : "2021-03-26 11:49:26.514",
"logLevel" : "INFO",
"IP" : "10.0.3.12",
"inout" : "-",
"Method" : "POST",
"url" : "https://api.kr-seo.assistant.watson.cloud.ibm.com/instances/a33da834-a7a7-48c2-9bf6-d3207849ad71/v1/workspaces/c6e3035b-411a-468d-adac-1ae608f7bf68/message?version=2018-07-10",
"status" : "200",
"duration" : "462 ms"

如果，inout 字段为 '+'，则状态/持续时间字段为空 ('')

我如何编写 logstash grok 过滤器的脚本？（grok，改变任何其他过滤器OK ...等）帮帮我...！

Answer 1

filter {

      grok { match => [ "message", "%{GREEDYDATA:predata} (?<inout>[-+]) \"%{GREEDYDATA:postdata}\""] }

      if [inout] == "+" 
        {
           grok { match => [ "message", "%{DATESTAMP:timestamp}: %{GREEDYDATA:data} \[%{LOGLEVEL:loglevel}\] %{IP:IP} (?<inout>[-+]) \"%{WORD:method} %{URI:url}\"" ] }
        }

      else {
          grok { match => [ "message", "%{DATESTAMP:timestamp}: %{GREEDYDATA:data} \[%{LOGLEVEL:loglevel}\] %{IP:IP} (?<inout>[-+]) \"%{WORD:method} %{URI:url}\" %{POSINT:statucode} %{POSINT:duration}" ] }
        }

   }

现在，您可以删除不需要的字段：


filter {
      mutate {
          remove_field => [
              "message",
              "predata",
              "postdata",
              "DATE_US",
              "IPV6",
              "USER",
              "USERNAME",
              "URIHOST",
              "IPORHOST",
              "HOSTNAME",
              "URIPATHPARAM",
              "port",
              "URIPATH",
              "URIPARAM"
          ]
          remove_tag => [
              "multiline",
              "_grokparsefailure"
          ]
      }
  }

如何使用logstash转换日志消息？

1 个答案: