Question

我有以下配置来获取Http调用和日期调用时的日志日期，但@timestamp显示日志从FileBeat服务器发送到logstash一个。

 input {
        beats {
                port => 5044
        }
}

filter {
         if [message] !~ /Calling SMC REST API/ {
                 drop { }
        }else {
                grok {
                        match => ["message", "{URIPATHPARAM:request}"]
                        match => ["message", "\[%{DATA:timestamp}\]%{GREEDYDATA:extra} %{GREEDYDATA:date}"] 
                }

                date {
                        match => ["timestamp", "[YYYY-MM-dd HH:mm:ss,SSS]"]
                        target => "timestamp"
                }

        }


}

output {
        elasticsearch {
                hosts => ["http://localhost:9200/"]
                index => "smc_calls-%{+YYYY.MM.dd}"

        }

stdout { codec => rubydebug }
}

输入日志消息：

[2017-04-25 12：41：25,064] rf234234232345454：Sg2343 INFO emcJmsContainer-2 - ******呼叫 SMC REST API ******** http://com.asds.dsds:45/api?search=query

红宝石输出：

"date" => "http-call", --> dont why http call is shown here
        "offset" => 18036878,
    "input_type" => "log",
        "source" => "log source",
       "message" => "entire filtered message",
          "type" => "log",
          "tags" => [
        [0] "beats_input_codec_plain_applied",
        [1] "_dateparsefailure"
    ],
    "@timestamp" => 2017-05-03T06:08:59.486Z,
         "extra" => "reaminaing data from message",
      "@version" => "1",
          "beat" => {
        "hostname" => "host",
            "name" => "host",
         "version" => "5.3.1"
    },
          "host" => "icgaspadsw01d",
     "timestamp" => "2017-04-25 12:41:25,064"  --> log timestamp i want but dont see in kibana when indexed

Answer 1

我认为你的grok解析器是以错误的方式编写的。在grok中使用它

\[%{DATA:timestamp}\]%{GREEDYDATA:extra} %{GREEDYDATA:date}

然后使用您的日期过滤器。

Answer 2

使用以下grok加日期过滤器：

grok {
                        match => ["message", "{URIPATHPARAM:request}"]
                        match => ["message", "%{TIMESTAMP_ISO8601:time}"]
                }

                 date {
                        match => ["time", "YYYY-MM-dd HH:mm:ss,SSS"]
                }

时间戳没有使用日期过滤器logstash获取我的日志值？

2 个答案: