将log4net Level字段添加到logstash.conf文件

时间:2019-03-29 18:33:08

标签: logstash logstash-configuration

我正在尝试添加LEVEL字段(因此它显示在Kibana中)。我的logstash.conf

输入:

2018-03-18 15:43:40.7914 - INFO: Tick 
2018-03-18 15:43:40.7914 - ERROR: Tock

文件:

input {
  beats {
    port => 5044
  }
}
filter {
  grok {      
      match => { 
            "message" => "(?m)^%{TIMESTAMP_ISO8601:timestamp}~~\[%{DATA:thread}\]~~\[%{DATA:user}\]~~\[%{DATA:requestId}\]~~\[%{DATA:userHost}\]~~\[%{DATA:requestUrl}\]~~%{DATA:level}~~%{DATA:logger}~~%{DATA:logmessage}~~%{DATA:exception}\|\|"
        }
      match => {
        "levell" => "(?m)^%{DATA:level}"
      }
      add_field => { 
        "received_at" => "%{@timestamp}" 
        "received_from" => "%{host}"
        "level" => "levell"
      }
      remove_field => ["message"]      
    }
  date {
    match => [ "timestamp", "yyyy-MM-dd HH:mm:ss:SSS" ]
  }
}
output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    sniffing => true
    index => "filebeat-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
    #user => "elastic"
    #password => "changeme"
  }
  stdout { codec => rubydebug }
}

这会打印出“ levell”,而不是“ INFO / ERROR”等

编辑: 输入:

2018-03-18 15:43:40.7914 - INFO: Tick 

配置:

# Sample Logstash configuration for creating a simple
# Beats -> Logstash -> Elasticsearch pipeline.

input {
  beats {
    port => 5044
  }
}
filter {
  grok {      
      match => { "message" => "(?m)^%{TIMESTAMP_ISO8601:timestamp}~~\[%{DATA:thread}\]~~\[%{DATA:user}\]~~\[%{DATA:requestId}\]~~\[%{DATA:userHost}\]~~\[%{DATA:requestUrl}\]~~%{DATA:level}~~%{DATA:logger}~~%{DATA:logmessage}~~%{DATA:exception}\|\|" }
      add_field => { 
        "received_at" => "%{@timestamp}" 
        "received_from" => "%{host}"
      } 
    }
  grok {      
      match => { "message" => "- %{LOGLEVEL:level}" }
      remove_field => ["message"]      
    }
  date {
    match => [ "timestamp", "yyyy-MM-dd HH:mm:ss:SSS" ]
  }
}
output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    sniffing => true
    index => "filebeat-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
    #user => "elastic"
    #password => "changeme"
  }
  stdout { codec => rubydebug }
}

我得到的输出。仍缺少Received_at和级别: enter image description here

1 个答案:

答案 0 :(得分:1)

在配置的那一部分:

  add_field => { 
    "received_at" => "%{@timestamp}" 
    "received_from" => "%{host}"
    "level" => "levell"
  }

使用"level" => "levell"时,只需将字符串levell放在字段level中。要放置名为levell的字段的值,必须使用%{levell}。因此,在您的情况下,它看起来像:

  add_field => { 
    "received_at" => "%{@timestamp}" 
    "received_from" => "%{host}"
    "level" => "%{levell}"
  }

根据documentationgrok#match

  

一个散列,该散列定义了查找位置和模式的映射。

因此尝试在levell字段上进行匹配将不起作用,因为它似乎尚不存在。您用来匹配message字段的grok模式与您提供的示例不匹配。