Question

[clientip]的映射器与其他类型的现有映射冲突：\ n [mapper [clientip]具有不同的[规范]值，无法从禁用更改为启用]

Elasticsearch 6.2.4

我们使用logstash + elasticsearch来获取我们的云端使用情况的一些带宽指标。这就是logstash解析事物的方式：https://gist.github.com/chrisan/1c5ce5beacfc0e124d39fa842f051857#file-logstash-api-conf

这会生成以下索引：https://gist.github.com/chrisan/1c5ce5beacfc0e124d39fa842f051857#file-indicies

使用以下映射：https://gist.github.com/chrisan/1c5ce5beacfc0e124d39fa842f051857#file-mappings

我被要求获取不同的IP地址，我尝试使用聚合查询：

{
  "size": 0,
    "aggs" : {
        "distinct_ips" : {
            "filter" : { "term": { "company" : "XXX" } },
            "aggs" : {
                "cardinality" : { "cardinality": {"field": "clientip"   } }
            }
        }
    }
}

但是这会回来：

{
    "error": {
        "root_cause": [
            {
                "type": "illegal_argument_exception",
                "reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [clientip] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."
            }
        ],
        "type": "search_phase_execution_exception",
        "reason": "all shards failed",
        "phase": "query",
        "grouped": true,
        "failed_shards": [
            {
                "shard": 0,
                "index": "logstash-2018.01.01",
                "node": "dO1JCnAnSmmk5EfDmfYgqQ",
                "reason": {
                    "type": "illegal_argument_exception",
                    "reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [clientip] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."
                }
            }
        ]
    },
    "status": 400
}

所以我尝试用

更新它

PUT / * / _ mapping / _doc？update_all_types

{
  "properties": {
    "clientip": {
      "type":     "text",
      "fielddata": true
    }
  }
}

返回：

{
    "error": {
        "root_cause": [
            {
                "type": "illegal_argument_exception",
                "reason": "Mapper for [clientip] conflicts with existing mapping in other types:\n[mapper [clientip] has different [norms] values, cannot change from disable to enabled]"
            }
        ],
        "type": "illegal_argument_exception",
        "reason": "Mapper for [clientip] conflicts with existing mapping in other types:\n[mapper [clientip] has different [norms] values, cannot change from disable to enabled]"
    },
    "status": 400
}

我做错了什么？

Answer 1

想出来，必须删除所有内容，然后使用

更新logstash模板

{
        "template": "logstash",
        "order": 0,
        "version": 60001,
        "index_patterns": [
            "logstash-*"
        ],
        "settings": {
            "index": {
                "refresh_interval": "5s"
            }
        },
        "mappings": {
            "_default_": {
                "dynamic_templates": [
                    {
                        "message_field": {
                            "path_match": "message",
                            "match_mapping_type": "string",
                            "mapping": {
                                "type": "text",
                                "norms": false
                            }
                        }
                    },
                    {
                        "string_fields": {
                            "match": "*",
                            "match_mapping_type": "string",
                            "mapping": {
                                "type": "text",
                                "norms": false,
                                "fields": {
                                    "keyword": {
                                        "type": "keyword",
                                        "ignore_above": 256
                                    }
                                }
                            }
                        }
                    }
                ],
                "properties": {
                    "@timestamp": {
                        "type": "date"
                    },
                    "@version": {
                        "type": "keyword"
                    },
                    "clientip": {
                        "type":     "text",
                        "fields": {
                        "keyword": { 
                          "type": "keyword"
                        }
                      }
                    },
                    "company": { 
                      "type": "text",
                      "fields": {
                        "keyword": { 
                          "type": "keyword"
                        }
                      }
                    },
                    "geoip": {
                        "dynamic": true,
                        "properties": {
                            "ip": {
                                "type": "ip"
                            },
                            "location": {
                                "type": "geo_point"
                            },
                            "latitude": {
                                "type": "half_float"
                            },
                            "longitude": {
                                "type": "half_float"
                            }
                        }
                    }
                }
            }
        },
        "aliases": {}
    }

然后我可以用

查询

"aggs": {
   "distinct_ips": {
     "cardinality": {
       "field": "clientip.keyword"
     }
   }
 }

mapper [clientip]有不同的[规范]值，无法从禁用更改为启用]

1 个答案: