如何将表单MYSQL导入Elasticsearch

时间:2016-09-28 14:44:25

标签: mysql jdbc elasticsearch

我正在尝试使用JDBC插件将MySQL数据库导入ES。我在嵌套字段方面遇到了一些问题:

我的映射是这样的:

{
    "product":{
        "properties":{
            "id":{ "type":"integer", "index": "not_analyzed" },
            "name":{ "type":"string" },
            "images":{
                "type":"nested",
                "properties": {
                    "id":{ "type":"integer","index": "no" },
                    "src":{ "type":"string","index": "not_analyzed" },
                    "sort_position":{ "type":"integer","index": "not_analyzed" }
                }
            }
        }
    }
}

我在SQL查询中使用GROUP_CONCAT和JSON_OBJECT,但弹性搜索无法识别。

bin=$JDBC_IMPORTER_HOME/bin
lib=$JDBC_IMPORTER_HOME/lib
echo '{
    "type" : "jdbc",
    "jdbc" : {
        "elasticsearch.autodiscover":true,
        "elasticsearch.cluster":"elasticsearch",
        "url" : "jdbc:mysql://myip:3306/modomu_tipo",
        "user" : "root",
        "password" : "123456",
        "sql" : [
            {
                "statement" : "SELECT p.id _id, CONCAT(\"{\",GROUP_CONCAT(DISTINCT(JSON_OBJECT(\"images.src\",images.src))),\"}\") as `images[]`, p.name FROM products p LEFT JOIN products_images images ON images.product_id=p.id GROUP BY p.id ORDER BY id LIMIT 10",
                "parameter" : []
            }
        ],
        "index" : "myindex",
        "type" : "product",
        "elasticsearch": {
             "host" : "myip",
             "port": 9300
        }
    }
}' | java \
       -cp "/root/elasticsearch-jdbc-2.3.4.1/lib/*" \
       -Dlog4j.configurationFile=/root/elasticsearch-jdbc-2.3.4.1/bin/log4j2.xml \
       org.xbib.tools.Runner \
       org.xbib.tools.JDBCImporter

这是我的日志错误:

[15:52:28,499][ERROR][org.xbib.elasticsearch.helper.client.BulkTransportClient][elasticsearch[importer][listener][T#1]] bulk [1] failed with 10 failed items, failure message = failure in bulk execution:
[0]: index [myindex], type [product], id [292418], message [MapperParsingException[object mapping for [images] tried to parse field [images] as object, but found a concrete value]]
[1]: index [myindex], type [product], id [292419], message [MapperParsingException[object mapping for [images] tried to parse field [images] as object, but found a concrete value]]
[2]: index [myindex], type [product], id [292420], message [MapperParsingException[object mapping for [images] tried to parse field [images] as object, but found a concrete value]]
[3]: index [myindex], type [product], id [292421], message [MapperParsingException[object mapping for [images] tried to parse field [images] as object, but found a concrete value]]
[4]: index [myindex], type [product], id [292422], message [MapperParsingException[object mapping for [images] tried to parse field [images] as object, but found a concrete value]]
[5]: index [myindex], type [product], id [292423], message [MapperParsingException[object mapping for [images] tried to parse field [images] as object, but found a concrete value]]
[6]: index [myindex], type [product], id [292424], message [MapperParsingException[object mapping for [images] tried to parse field [images] as object, but found a concrete value]]
[7]: index [myindex], type [product], id [292425], message [MapperParsingException[object mapping for [images] tried to parse field [images] as object, but found a concrete value]]
[8]: index [myindex], type [product], id [292426], message [MapperParsingException[object mapping for [images] tried to parse field [images] as object, but found a concrete value]]
[9]: index [myindex], type [product], id [292427], message [MapperParsingException[object mapping for [images] tried to parse field [images] as object, but found a concrete value]]

提前感谢您的帮助。

0 个答案:

没有答案