创建索引映射后，ES索引速率变慢

Question

我使用ES BulkProcessor写入数据（我尝试使用python脚本，storm es-bolt，flink es-sink），但是创建索引映射后，索引速率是如此之慢。

• 情况1：将所有索引设置保留为默认设置，索引率可以达到10000 +。
• 情况2：仅创建索引映射，索引率降至3000。

我使用相同的数据，相同的代码，相同的机器。

结果

flink es-sink将json数据写入es：

我的数据

在下面重复写入相同的数据（message字段是原始日志，大约7KB的大小，删除一些超出问题限制的内容）：

    {
        "_index": "nyc_flink_test997",
        "_type": "doc",
        "_id": "k8uS92cBOH4ugSIjCzmn",
        "_score": 1,
        "_source": {
          "exception": "false",
          "log_id": "8F71AF1606EE46BFA9D57AA2282D8596",
          "offset": "2368",
          "message_length": "2103",
          "level": "INFO",
          "source": "/opt/hadoop/elastic-stack/s_login/Gusermanager.usermanager.s_login.20.log",
          "sessionid": "provider-60-2883b4bd3ff2b",
          "associate_id": "33d081b83a0654a2",
          "message": """
          [16:41:33.376][I][ec4edfe0b2584b73]log start:53F9A1A1E71044E281755E930E1B004C
          [16:41:33.376][T][ec4edfe0b2584b73]入参0=__REQ__
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4119)
    at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2570)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2731)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2815)
    at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2155)
    at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:2322)
    at cn.com.agree.addal.cp.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:46)
    at tc.bank.aesb.mbs.MBS_DBIMPL.PyDBGetSel(MBS_DBIMPL.java:1624)
    at tc.bank.aesb.mbs.MBS_DBIMPL.PyDBExecOneSQL(MBS_DBIMPL.java:466)
    at tc.bank.aesb.mbs.MBS_DBIMPL.PyDBExecGrpSQL(MBS_DBIMPL.java:123)
    at tc.bank.aesb.mbs.B_MBS_DataBase.B_DBUnityRptOpr(B_MBS_DataBase.java:121)
    at CUST.CustomerInfoQry.TCustomerInfoQry$Step1$Node4.execute(TCustomerInfoQry.java:200)
    at CUST.CustomerInfoQry.TCustomerInfoQry$Step1.execute(TCustomerInfoQry.java:113)
    at CUST.CustomerInfoQry.TCustomerInfoQry.execute(TCustomerInfoQry.java:76)
    at cn.com.agree.afa.svc.javaengine.JavaEngine.execute(JavaEngine.java:237)
    at cn.com.agree.afa.svc.handler.TradeHandler.handle(TradeHandler.java:62)
[16:41:33.414][I][ec4edfe0b2584b73]log end:53F9A1A1E71044E281755E930E1B004C
""",
          "exec_ip": "10.88.188.167",
          "start_time": "2018-12-09 16:46:14.764",
          "group_v2": "Gusermanager",
          "script_exec_time": "1",
          "trade_exec_time": "2"
        }
}

索引映射

{
      "mappings": {
        "doc":{
            "dynamic_templates": [
                {
                    "string_fields": {
                        "match": "*",
                        "match_mapping_type": "string",
                        "mapping": {
                            "type": "text",
                            "norms": false,
                            "fields": {
                                "keyword": {
                                    "type": "keyword",
                                    "ignore_above": 256
                                }
                            }
                        }
                    }
                }
            ],
            "properties": {
                "@timestamp": {
                    "type": "date"
                },
                "@version": {
                    "type": "keyword"
                },
                "geoip": {
                    "dynamic": true,
                    "properties": {
                        "ip": {
                            "type": "ip"
                        },
                        "location": {
                            "type": "geo_point"
                        },
                        "latitude": {
                            "type": "half_float"
                        },
                        "longitude": {
                            "type": "half_float"
                        }
                    }
                },
                "exception": {
                    "type": "boolean"
                },
                "message":{
                    "type":"text",
                    "norms": false,
            "analyzer": "ik_max_word"
                },
                "associate_id": {
                    "type": "text",
                    "analyzer": "ik_max_word"
                },
                "end_time": {
                    "type": "date",
                "format": "date_time||yyyy-MM-dd HH:mm:ss.SSS||yyyy-MM-dd||epoch_millis||HH:mm:ss.SSS"
                },
                "start_time": {
                    "type": "date",
                "format": "date_time||yyyy-MM-dd HH:mm:ss.SSS||yyyy-MM-dd||epoch_millis||HH:mm:ss.SSS"
                },

                "exec_ip": {
                    "type": "ip"
                },
                "level": {
                    "type": "keyword"
                },
                "script_exec_time": {
                    "type": "long"
                },
                "trade_exec_time": {
                    "type": "long"
                },
                "sessionid": {
                    "type": "text",
                    "analyzer": "ik_max_word"
                },
                "log_id": {
                    "type": "text",
                    "analyzer": "ik_max_word"
                },
                "discard_time": {
                    "type": "long"
                },
                "scene_code": {
                    "type": "text",
                    "analyzer": "ik_max_word"
                },
                "service_code": {
                    "type": "text",
                    "analyzer": "ik_max_word"
                },
                "group": {
                    "type": "text"
                },
                "group_v2" :{
                    "type": "text",
                    "analyzer": "ik_max_word"
                },
                "message_length":{
                    "type": "long"
                },
                "log_filename":{
                    "type": "text",
                    "analyzer": "ik_max_word"
                },
                "ingest_time":{
                    "type": "date"
                }
            }
        }
    }
}

我尝试使用python脚本编写脚本，风暴es-bolt，结果相同，创建索引映射后索引率下降。任何人都可以提出一些想法。预先感谢。