Question

我正在尝试使用以下Logstash脚本（ELK v 6.22）将数据从MySQL导入Elastic索引：

input {
  jdbc {
    jdbc_driver_library => "E:\ELK 6.22\logstash-6.2.2\bin\mysql-connector-java-5.1.45-bin.jar"
    jdbc_driver_class => "com.mysql.jdbc.Driver"
    jdbc_connection_string => "jdbc:mysql://localhost:3306/fbk"
    jdbc_user => "root"
    jdbc_password => ""
    statement => "SELECT fbk_repeat._URI AS URI, _SUBMISSION_DATE AS SUBMISSION_DATE, DEVICEID, LOCATION_LAT, LOCATION_LNG, SECTOR, COMMENTS, ACTION_TAKEN, PURPOSE
                    FROM
                    fbk_core
                    INNER JOIN fbk_repeat ON fbk_core._URI = fbk_repeat._PARENT_AURI"
  }
}



filter {

#    mutate { convert => {"LOCATION_LAT" => "float"} }
#    mutate { convert => {"LOCATION_LNG" => "float"} }
#    mutate { rename => {"LOCATION_LAT" => "[location][lat]"} }
#    mutate { rename => {"LOCATION_LNG" => "[location][lon]"} }

    mutate {
        # Location and lat/lon should be used as is, this is as per logstash documentation
        # Here we are tying to create a two-dimensional array in order to save data as per Logstash documentation

         add_field => { "[location][lat]" => [ "%{LOCATION_LAT}" ] }
         add_field => { "[location][lon]" => [ "%{LOCATION_LNG}" ] }
         convert => [ "[location]", "float" ]
    }


#    date {
#        locale => "eng"
#        match => ["_SUBMISSION_DATE", "yyyy-MM-dd HH:mm:ss", "ISO8601"]
#        target => "SUBMISSION_DATE"
#           } 

}

output{
   elasticsearch {
   hosts => ["localhost:9200"] 
   index => "feedback"
   document_id => "%{URI}"
   document_type => "feedbackdata"
   manage_template => true

   # user => "elastic"
   # password => "changeme"


}
  stdout { codec => rubydebug { metadata => true } }
   # stdout { codec => dots }
}

导入数据后，我无法在Kibana找到任何Geo Point字段，以便能够将数据绘制到地图中，任何人都可以指导必须出错的地方。

谢谢！

数据

Answer 1

Elasticsearch可以自动执行映射，但不能自动执行映射。您应该像这样设置映射：

PUT index
{
  "mappings": {
    "type": {
      "properties": {
        "location": {
          "properties": {
            "coordinates": {
              "type": "geo_point"
            }
          }
        },
        "field": {
          "properties": {
            "date": {
              "format": "yyyy-MM-dd'T'HH:mm:ss.SSSZ",
              "type": "date"
            }
          }
        }
      }
    }
  }
}

对此进行调整以处理您的数据。

不要忘记在Kibana中创建index pattern。

将Geo Point（Lat，Lng）从MySQL导入Elasticsearch

1 个答案: