Question

我有一个简单的Logstash 1.4.1设置，数据在stdout中似乎是正确的，但在ES / Kibana中，＆＃34;消息＆＃34;字段不会被覆盖。

logstash.conf

input {
  file {
    path => "/var/log/nginx/access.log"
    type => "nginx_access"
  }
}

filter {
  if [type] == "nginx_access" {
    grok {
      match => [ "message", "%{NGINXACCESS}" ]
      overwrite => [ "message" ]
    }
  }
}

output {
  elasticsearch {
    protocol => "http"
  }
  stdout { codec => rubydebug }
}

NGINX模式

NGUSERNAME [a-zA-Z\.\@\-\+_%]+
NGUSER %{NGUSERNAME}
NGINXACCESS %{IPORHOST:clientip} %{NGUSER:ident} %{NGUSER:auth} \[%{HTTPDATE:timestamp}\] "%{WORD:verb} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response} (?:%{NUMBER:bytes}|-) (?:"(?:%{URI:referrer}|-)"|%{QS:referrer}) %{QS:agent}

标准输出

{
        "message" => "127.0.0.1 - - [27/Nov/2014:13:43:53 +0100] \"GET /favicon.ico HTTP/1.1\" 502 574 \"-\" \"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.122 Safari/537.36\"",
       "@version" => "1",
     "@timestamp" => "2014-11-27T12:43:54.230Z",
           "type" => "nginx_access",
           "host" => "laptop",
           "path" => "/var/log/nginx/access.log",
       "clientip" => "127.0.0.1",
          "ident" => "-",
           "auth" => "-",
      "timestamp" => "27/Nov/2014:13:43:53 +0100",
           "verb" => "GET",
        "request" => "/favicon.ico",
    "httpversion" => "1.1",
       "response" => "502",
          "bytes" => "574",
          "agent" => "\"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.122 Safari/537.36\""
}

我做错了什么？我该如何调试？

修改

Get API显示这些字段位于_source中，未编入索引。怎么办？

{  
   "_index":"logstash-2014.11.27",
   "_type":"nginx_access4",
   "_id":"AUnxR34Z5DzKTB-2TeuG",
   "_version":1,
   "found":true,
   "_source":{  
      "message":"127.0.0.1 - - [27/Nov/2014:13:43:53 +0100] \"GET / HTTP/1.1\" 502 574 \"-\" \"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.122 Safari/537.36\"",
      "@version":"1",
      "@timestamp":"2014-11-27T12:43:54.230Z",
      "type":"nginx_access",
      "host":"laptop",
      "path":"/var/log/nginx/access.log",
      "clientip":"127.0.0.1",
      "ident":"-",
      "auth":"-",
      "timestamp":"27/Nov/2014:13:43:53 +0100",
      "verb":"GET",
      "request":"/",
      "httpversion":"1.1",
      "response":"502",
      "bytes":"574",
      "agent":"\"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.122 Safari/537.36\""
   }
}

Answer 1

重新加载我的索引模式字段列表有帮助。我在记录任何数据之前创建了那个。

Logstash Grokked数据无法访问ElasticSearch

1 个答案: