Question

我的logstasg配置让我发疯了。

我找不到用另一种方法替换@timestamp字段的方法：

以下是logstash收到的内容：

{
                          "offset" => 6718968,
                    "Varnish_txid" => "639657758",
                       "plateform" => "cdnfronts",
                         "Referer" => "-",
                      "input_type" => "log",
                        "respsize" => "281",
                          "source" => "/var/log/varnish/varnish4xx-5xx.log",
                              "UA" => "Microsoft-WebDAV-MiniRedir/5.1.2600",
                            "type" => "varnish-logs",
                            "tags" => [
        [0] "json",
        [1] "varnish",
        [2] "beats_input_codec_json_applied",
        [3] "_dateparsefailure"
    ],
            "st_snt2c_or_sntfromb" => "405",
                      "RemoteHost" => "32.26.21.21",
                      "@timestamp" => 2017-02-14T13:38:47.808Z,
                "Varnish.Handling" => "pass",
    "tot_bytes_rcvby_c_or_sntby_b" => "-",
         "time_req_rcv4c_or_snt4b" => "[14/Feb/2017:14:38:44 +0100]",
                        "@version" => "1",
                            "beat" => {
        "hostname" => "cdn1",
            "name" => "cdn1",
         "version" => "5.1.2"
    },
                            "host" => "cdn1",
                   "time_1st_byte" => "0.010954",
                    "Varnish_side" => "c",
                    "reqfirstline" => "OPTIONS http://a.toto.com/ HTTP/1.1"
}

这是我的logstash conf：

input {
    beats {
        port => 5000
        codec => "json"
        ssl => true
        ssl_certificate => "/etc/logstash/ssl/logstash-forwarder.crt"
        ssl_key => "/etc/logstash/ssl/logstash-forwarder.key"

    }
}
filter {
  if "json" in [tags] {
    json {
      source => "message"
    }
    if "varnish" in [tags] {
     date {
       locale => "en"
       match => [ "[time_req_rcv4c_or_snt4b]","dd/MMM/yyyy:HH:mm:ss Z" ]
       remove_field => "[time_req_rcv4c_or_snt4b]"
      }
    }
  }
}
output {
  if "varnish" in [tags] {
      elasticsearch {
        hosts => ["elasticsearch:9200"]
        index => "logstash-varnish-%{+YYYY.MM.dd}"
    }
  } else {
      elasticsearch {
        hosts => ["elasticsearch:9200"]
      }
  }

    stdout {
        codec => rubydebug
    }
}

我试过了：

   match => [ "time_req_rcv4c_or_snt4b","dd/MMM/yyyy:HH:mm:ss Z" ]
   remove_field => "time_req_rcv4c_or_snt4b"

和

match => [ "[time_req_rcv4c_or_snt4b]","dd/MMM/yyyy:HH:mm:ss Z" ]
   remove_field => "[time_req_rcv4c_or_snt4]

任何人都可以解释我错过了什么。我暂时没有在谷歌上找到任何相关内容。

Answer 1

从你的输出：

"time_req_rcv4c_or_snt4b" => "[14/Feb/2017:14:38:44 +0100]",

您的日期字段周围有[]，因此您需要匹配日期模式中的日期，或者在第一次匹配日期时将其删除。

在logstash中替换@timestamp

1 个答案: