在Logstash中从CSV(纬度和经度列)创建地理点对象

时间:2016-06-23 22:26:40

标签: csv elasticsearch logstash kibana

我有一个包含latitudelongitude列的CSV,我试图在Logstash 2.3.3中创建一个geopoint对象,以便我可以在Kibana 4.5.1。

然而,当在Kibana中可视化数据时,我看到location.latlocation.lonfloat类型而非类型location的{​​{1}}。< / p>

我一般都是ELK的新手,这让我发疯。特别是因为我发现的大部分信息都已过时。

我使用的geopoint文件如下所示:

.conf

我指定(input { file { path => "C:/file.csv" start_position => "beginning" } } filter { csv { separator => "," columns => ["longitude","latitude"] } mutate { convert => {"latitude" => "float"} } mutate { convert => {"longitude" => "float"} } mutate { rename => {"latitude" => "[location][lat]"} } mutate { rename => {"longitude" => "[location][lon]"} } mutate { convert => { "[location]" => "float" } } } output { elasticsearch { template => "...\elasticsearch-template.json" template_overwrite => true action => "index" hosts => "localhost" index => "testindex1" workers => 1 } stdout {} } )的模板文件如下:

elasticsearch-template.json

如果有人能帮助我或者让我对我做错了什么有所了解,我将非常感激。此外,我确信这将有助于与我在同一条船上的所有人。

我解决了它,它现在运行得很好。模板正在寻找{ "template" : "logstash-*", "settings" : { "index.refresh_interval" : "5s" }, "mappings" : { "_default_" : { "_all" : {"enabled" : true, "omit_norms" : true}, "dynamic_templates" : [ { "message_field" : { "match" : "message", "match_mapping_type" : "string", "mapping" : { "type" : "string", "index" : "analyzed", "omit_norms" : true, "fielddata" : { "format" : "disabled" } } } }, { "string_fields" : { "match" : "*", "match_mapping_type" : "string", "mapping" : { "type" : "string", "index" : "analyzed", "omit_norms" : true, "fielddata" : { "format" : "disabled" }, "fields" : { "raw" : {"type": "string", "index" : "not_analyzed", "ignore_above" : 256} } } } } ], "properties" : { "@timestamp": { "type": "date" }, "@version": { "type": "string", "index": "not_analyzed" }, "geoip" : { "dynamic": true, "properties" : { "ip": { "type": "ip" }, "location" : { "type" : "geo_point" }, "latitude" : { "type" : "float" }, "longitude" : { "type" : "float" } } }, "location" : { "type": "geo_point" } } } } } 类型的索引,而我正在使用logstash-*。将我的索引更改为testindex1修复它。

1 个答案:

答案 0 :(得分:4)

您需要删除最后一个mutate过滤器,这会破坏您尝试实现的目的。

此外,您需要确保testindex1映射忠实地包含您在elasticsearch-template.json文件中的映射