Question

我尝试使用ELK堆栈解析一些Apache访问日志，但我遇到了logstash没有应用我在任何Apache日志上创建的Apache过滤器的问题。这是我的过滤文件：

filter {
  if [type] == "apache_access" {
    grok {
      patterns_dir => ["/opt/logstash/patterns/apache"]
      add_tag => ["grokked", "apache"]
      match => ["messege", "%{IP:client} - - \[%{HTTPDATE:event_date}\] %{QS:first} %{NUMBER:response} %{NUMBER:bytes} %{QS:destination} %{QS:browser}"]
    }
  }
}

filebeat config：

filebeat:
  prospectors:
    -
      paths:
        - /var/log/apache2/access.log
      document_type: apache_access
  registry_file: /var/lib/filebeat/registry

此外，我使用logz.io中的示例日志文件，它包含如下日志：

88.114.162.149 - - [04/Aug/2016:00:00:05 +0000] "GET /item/giftcards/3802 HTTP/1.1" 200 82 "/category/books" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0.1) Gecko/20100101 Firefox/9.0.1"
156.141.192.36 - - [04/Aug/2016:00:00:10 +0000] "GET /category/toys?from=20 HTTP/1.1" 200 135 "/category/toys" "Mozilla/5.0 (Windows NT 6.0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11"
92.213.110.215 - - [04/Aug/2016:00:00:15 +0000] "GET /category/software HTTP/1.1" 200 108 "/category/books" "Mozilla/5.0 (Windows NT 6.0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11"
80.225.119.24 - - [04/Aug/2016:00:00:20 +0000] "GET /category/cameras HTTP/1.1" 200 100 "http://www.google.com/search?ie=UTF-8&q=google&sclient=psy-ab&q=Cameras+Books&oq=Cameras+Books&aq=f&aqi=g-vL1&aql=&pbx=1&bav=on.2,or.r_gc.r_pw.r_qf.,cf.osb&biw=2640&bih=427" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; YTB730; GTB7.2; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET4.0C; .NET4.0E; Media Center PC 6.0)"
208.219.150.176 - - [04/Aug/2016:00:00:25 +0000] "GET /category/software HTTP/1.1" 200 117 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; GTB7.2; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C)"
160.165.186.172 - - [04/Aug/2016:00:00:30 +0000] "GET /category/office HTTP/1.1" 200 101 "/category/electronics" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; YTB720; GTB7.2; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"
224.150.219.97 - - [04/Aug/2016:00:00:35 +0000] "GET /category/jewelry HTTP/1.1" 200 74 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

我已经在grokdebug中检查了我的过滤器并且一切正常，但每次我将这些日志推入logstash它都不会应用该过滤器，而是所有日志条目都有＆＃34; _grokparsefailure＆＃ 34;标签

知道这里有什么问题吗？我已经关注了几位导游，但仍然遇到了这个问题。

P.S。我知道COMBINEDAPACHELOG但我仍然想以这种方式解析它以获得我自己的经验并理解ELK堆栈击球。

Answer 1

尝试在您的grok匹配中将messege更改为message

         change 'e' to 'a'
                 |
                 v
  match => ["message", "%{IP:client} - - \[%{HTTPDATE:event_date}\] %{QS:first} %{NUMBER:response} %{NUMBER:bytes} %{QS:destination} %{QS:browser}"]

Logstash未将过滤器应用于Apache日志

1 个答案: