Question

我使用logstash-logback-encoder将java日志文件发送到logstash，然后发送到elasticsearch。要解析java日志中的message，我使用以下过滤器来剖析消息

input {
  file {
    path => "/Users/MacBook-201965/Work/java/logs/oauth-logstash.log"
    start_position => "beginning"
    codec => "json"
  }
}

filter {
  if "EXECUTION_TIME" in [tags] {
    dissect {
      mapping => {
        "message" => "%{endpoint} timeMillis:[%{execution_time_millis}] data:%{additional_data}"
      }
    }
    mutate {
      convert => { "execution_time_millis" => "integer" }
    }
  }
}

output {
  elasticsearch { 
     hosts => "localhost:9200"
     index => "elk-%{+YYYY}"
     document_type => "log"
  }

  stdout {
    codec => json
  }
}

它剖析了消息，因此我可以获得execution_time_millis的价值。但是数据类型是字符串。我使用Kibana索引模式创建了索引。如何将execution_time_millis的数据类型更改为long？

以下是来自logback的示例json消息

{  
   "message":"/tests/{id} timeMillis:[142] data:2282||0:0:0:0:0:0:0:1",
   "logger_name":"com.timpamungkas.oauth.client.controller.ElkController",
   "level_value":20000,
   "endpoint":"/tests/{id}",
   "execution_time_millis":"142",
   "@version":1,
   "host":"macbook201965s-MacBook-Air.local",
   "thread_name":"http-nio-8080-exec-7",
   "path":"/Users/MacBook-201965/Work/java/logs/oauth-logstash.log",
   "@timestamp":"2018-01-04T11:20:20.100Z",
   "level":"INFO",
   "tags":[  
      "EXECUTION_TIME"
   ],
   "additional_data":"2282||0:0:0:0:0:0:0:1"
}{  
   "message":"/tests/{id} timeMillis:[110] data:2280||0:0:0:0:0:0:0:1",
   "logger_name":"com.timpamungkas.oauth.client.controller.ElkController",
   "level_value":20000,
   "endpoint":"/tests/{id}",
   "execution_time_millis":"110",
   "@version":1,
   "host":"macbook201965s-MacBook-Air.local",
   "thread_name":"http-nio-8080-exec-5",
   "path":"/Users/MacBook-201965/Work/java/logs/oauth-logstash.log",
   "@timestamp":"2018-01-04T11:20:19.780Z",
   "level":"INFO",
   "tags":[  
      "EXECUTION_TIME"
   ],
   "additional_data":"2280||0:0:0:0:0:0:0:1"
}

谢谢

Answer 1

如果您已经索引了文档，则在更改任何字段的数据类型后，您必须重新索引数据。

但是，您可以使用类似的内容将millis的类型从字符串更改为整数。（此处不支持long）

https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-convert

另外，如果要添加索引名称具有一些正则表达式模式的多个索引，请尝试在创建索引之前定义elasticsearch模板。这样，您可以事先定义索引格式，然后再开始索引。

Elasticsearch从过滤器剖析中更改字段类型

1 个答案: