Question

日志：

[2017-01-14 10:48:06,848][WARN ][index.search.slowlog.query] [yaswanth] [bank][0] took[27.8ms], took_millis[27], types[], stats[], search_type[QUERY_THEN_FETCH], total_shards[5], source[], extra_source[], 
[2017-01-14 10:48:06,851][WARN ][index.search.slowlog.query] [yaswanth] [bank][3] took[12.7ms], took_millis[33], types[], stats[], search_type[QUERY_THEN_FETCH], total_shards[5], source[], extra_source[],

我正在尝试获取TOOK字段大于15ms的日志。我使用了以下配置

input {
  file {
    path => "F:\logstash-2.4.0\logstash-2.4.0\pica.txt"
    start_position => "beginning"
  }
}

filter {
           grok {
                match => [ "message", "\[%{TIMESTAMP_ISO8601:TIMESTAMP}\]\[%{LOGLEVEL:LEVEL}%{SPACE}\]\[%{DATA:QUERY}\]%{SPACE}\[%{DATA:QUERY1}\]%{SPACE}\[%{DATA:INDEX-NAME}\]\[%{DATA:SHARD}\]%{SPACE}took\[%{DATA:TOOK}\],%{SPACE}took_millis\[%{DATA:TOOKM}\], types\[%{DATA:types}\], stats\[%{DATA:stats}\], search_type\[%{DATA:search_type}\], total_shards\[%{NUMBER:total_shards}\], source\[%{DATA:source_query}\], extra_source\[%{DATA:extra_source}\],"]
           }
if [TOOK] > 15ms {
    mutate {
      add_tag => "slowresponse"
    }
  } else {
    drop { }
  }
}
output {
stdout { codec => rubydebug }

}

错误是这样的：

[31mException in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash. {"exception"=>#<NoMethodError: undefined method `>' for nil:NilClass>, "backtrace"=>["(eval):123:in `initialize'", "org/jruby/RubyArray.java:1613:in `each'", "(eval):121:in `initialize'", "org/jruby/RubyProc.java:281:in `call'", "(eval):91:in `filter_func'", "F:/logstash-2.4.0/logstash-2.4.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/pipeline.rb:267:in `filter_batch'", "org/jruby/RubyArray.java:1613:in `each'", "org/jruby/RubyEnumerable.java:852:in `inject'", "F:/logstash-2.4.0/logstash-2.4.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/pipeline.rb:265:in `filter_batch'", "F:/logstash-2.4.0/logstash-2.4.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/pipeline.rb:223:in `worker_loop'", "F:/logstash-2.4.0/logstash-2.4.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/pipeline.rb:201:in `start_workers'"], :level=>:error}[0m
NoMethodError: undefined method `>' for nil:NilClass
     initialize at (eval):123
           each at org/jruby/RubyArray.java:1613
     initialize at (eval):121
           call at org/jruby/RubyProc.java:281
    filter_func at (eval):91
   filter_batch at F:/logstash-2.4.0/logstash-2.4.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/pipeline.rb:267
           each at org/jruby/RubyArray.java:1613
         inject at org/jruby/RubyEnumerable.java:852
   filter_batch at F:/logstash-2.4.0/logstash-2.4.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/pipeline.rb:265
    worker_loop at F:/logstash-2.4.0/logstash-2.4.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/pipeline.rb:223
  start_workers at F:/logstash-2.4.0/logstash-2.4.0/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/pipeline.rb:201

我想看到TOOK字段大于15ms的日志。我甚至试过给"15ms"。

感谢

Answer 1

我建议使用TOOKM字段而不是TOOK字段，因为它包含的数字值比字符串1更容易比较。您还需要使用TOOKM将mutate/convert转换为整数。

修改后的配置将帮助您实现所需目标：

input {
  file {
    path => "F:\logstash-2.4.0\logstash-2.4.0\pica.txt"
    start_position => "beginning"
  }
}

filter {
    grok {
       match => [ "message", "\[%{TIMESTAMP_ISO8601:TIMESTAMP}\]\[%{LOGLEVEL:LEVEL}%{SPACE}\]\[%{DATA:QUERY}\]%{SPACE}\[%{DATA:QUERY1}\]%{SPACE}\[%{DATA:INDEX-NAME}\]\[%{DATA:SHARD}\]%{SPACE}took\[%{DATA:TOOK}\],%{SPACE}took_millis\[%{DATA:TOOKM}\], types\[%{DATA:types}\], stats\[%{DATA:stats}\], search_type\[%{DATA:search_type}\], total_shards\[%{NUMBER:total_shards}\], source\[%{DATA:source_query}\], extra_source\[%{DATA:extra_source}\],"]
    }

    # ==> add this filter to convert TOOKM to integer
    mutate {
        convert => { "TOOKM" => "integer" }
    }

    # ==> use TOOKM field instead
    if [TOOKM] > 15 {
        mutate {
            add_tag => "slowresponse"
        }
    } else {
        drop { }
    }
}
output {
   stdout { codec => rubydebug }
}

使用logstash获取具有大于15ms的TOOK字段的慢速日志？

1 个答案: