我已经在我的机器上安装了ELK堆栈。 Kibana在5.2.1中的版本，弹性版本是5.2.1。所以基本上一切都是最新的。 但现在我得到了警告。
“映射冲突！2个字段被定义为匹配此模式的索引中的几种类型（字符串，整数等）。您可能仍然可以在Kibana的部分中使用这些冲突字段，但它们将不适用于要求Kibana知道其类型的功能。纠正此问题需要重新编制数据索引。“
这两个字段是“ geoip.coordinates ”和“ geoip.location ”
我在谷歌上搜索解决方案，得到的结果如“ReIndex数据”。但我是ELK的新手，不知道如何重新编制索引
这是我的 template.json 文件，
{
"mappings": {
"_default_": {
"_all": {
"enabled": true,
"norms": {
"enabled": false
}
},
"dynamic_templates": [
{
"template1": {
"mapping": {
"doc_values": true,
"ignore_above": 1024,
"index": "not_analyzed",
"type": "{dynamic_type}"
},
"match": "*"
}
}
],
"properties": {
"@timestamp": {
"type": "date"
},
"message": {
"type": "string",
"index": "analyzed"
},
"offset": {
"type": "long",
"doc_values": true
},
"geoip" : {
"type" : "object",
"dynamic": true,
"properties" : {
"location" : { "type" : "geo_point" },
"coordinates" : { "type" : "geo_point" }
}
}
}
}
},
"settings": {
"index.refresh_interval": "5s"
},
"template": "filebeat-*"
}
和我的logstash.conf文件的输出部分
output {
elasticsearch {
hosts => ["localhost:9200"]
sniffing => true
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
我不知道解决方案应该是什么。任何帮助将不胜感激
答案 0 :(得分：0)
在elasticsearch控制台中编写GET /_cat/indices?v命令以获取有关所有索引的信息。 ＆＃34; kibana - ＆gt;管理 - ＆gt;索引模式＆＃34;没有告诉你所有的索引。它只是告诉您已配置的索引模式。
答案 1 :(得分：0)
当您将新字段添加到kibana时，您最常决定它是哪种类型。 之后 - 您将尝试编辑/etc/logstash/conf.d/filters.conf：
filter {
if [headers][request_path] == "/stats/" {
...
mutate {
remove_field => "[headers]"
convert => {
"intCourseId" => "integer"
"intLessonId" => "integer"
"intQuestionId" => "integer"
...
}
}
}
}
当您遵循此模式时，您将不会发生冲突。
否则，您可以修复elasticsearch索引中不正确的字段值。 Kibana将重新编制其明天的索引。 我在ruby上的脚本做到了：
#!/usr/bin/env ruby
# find and replace incorrect field values in elasticsearch indexes.
# Run:
# ruby ./bin/fix_elk_attribute_conflict elk_user=<name> elk_password=<password> index_type=<index_type> host=<host_ip>
# port=<host_port> attribute_name=<attribute_name> attribute_check_regexp='^\d+$'
# name, password - nginx basic auth credentials
# index_type=<index_type> - type of elasticsearch indexes
# host=<host_ip> port=<host_port> - host&port of elasticsearch node
# attribute_name=<attribute_name> - name of field which need to correct
# attribute_check_regexp - regexp for check if value is String
#
require 'net/http'
require 'uri'
require 'json'
command_line_arguments = Hash[ARGV.map { |x| x.split('=') }]
USER_NAME = command_line_arguments['elk_user']
USER_PASSWORD = command_line_arguments['elk_password']
INDEX_NAME = command_line_arguments['index_type']
HOST = command_line_arguments['host']
PORT = command_line_arguments['port']
ATTRIBUTE_NAME = command_line_arguments['attribute_name']
ATTRIBUTE_CHECK_REGEXP = Regexp.new(command_line_arguments['attribute_check_regexp'])
def update_document(index_id:, doc_id:, request_body:)
url = "http://#{USER_NAME}@#{HOST}:#{PORT}/#{index_id}/#{INDEX_NAME}/#{doc_id}"
puts url: url
puts request_body: request_body
uri = URI.parse(url)
request = Net::HTTP::Put.new(uri)
request['Content-Type'] = 'application/json'
request.basic_auth USER_NAME, USER_PASSWORD
request.body = JSON.dump(request_body) if request_body
response = Net::HTTP.start(uri.hostname, uri.port) do |http|
http.request(request)
end
data = JSON.parse(response.body)
puts request_complete_with_massage: data
end
def find_all_documents(command:, request_body: nil)
uri = URI.parse("http://#{USER_NAME}@#{HOST}:#{PORT}#{command}")
request = Net::HTTP::Get.new(uri)
request['Content-Type'] = 'application/json'
request.basic_auth USER_NAME, USER_PASSWORD
request.body = JSON.dump(request_body) if request_body
response = Net::HTTP.start(uri.hostname, uri.port) do |http|
http.request(request)
end
data = JSON.parse(response.body)
{
result: (data['hits'] || {})['hits'] || [],
total: (data['hits'] || {})['total']
}
end
doc_pointer = 0
page_size = 100
total_count = page_size + 100
total_count_message = true
errors = []
while doc_pointer < total_count do
documents = find_all_documents(command: '/_search?pretty', request_body: {
from: doc_pointer,
size: page_size,
_source: [:_id, "#{ATTRIBUTE_NAME}"],
query: {
exists: {
field: "#{ATTRIBUTE_NAME}"
}
}
})
if total_count_message
puts "Found #{documents[:total]} documents with attribute \"#{ATTRIBUTE_NAME}\"."
puts
total_count = documents[:total]
total_count_message = false
end
printf "\r Reading documents #{doc_pointer} - #{doc_pointer + page_size}"
documents[:result].each do |doc|
attribute_value = doc['_source'][ATTRIBUTE_NAME]
if !attribute_value
errors << {
attribute_value: attribute_value,
doc: doc['_id'],
index: doc['_index'],
message: "Document #{doc['_id']} have empty #{ATTRIBUTE_NAME} == '#{attribute_value}'",
correct_attribute_value: '0',
}
elsif !attribute_value.is_a? String
errors << {
attribute_value: attribute_value,
doc: doc['_id'],
index: doc['_index'],
message: "Document #{doc['_id']} have not string #{ATTRIBUTE_NAME} == '#{attribute_value}'",
correct_attribute_value: attribute_value.to_s,
}
elsif !attribute_value[ATTRIBUTE_CHECK_REGEXP]
errors << {
attribute_value: attribute_value,
doc: doc['_id'],
index: doc['_index'],
message: "Document #{doc['_id']} contains incorrect #{ATTRIBUTE_NAME} == '#{attribute_value}'",
correct_attribute_value: '0',
}
end
end
doc_pointer += page_size
end
puts
if errors.size > 0
puts "============ Errors found ============="
errors.each do |error, index|
puts index
puts error[:message]
puts doc: error[:doc]
puts index: error[:index]
puts ATTRIBUTE_NAME: ATTRIBUTE_NAME
puts attribute_value: error[:attribute_value]
fix_hash = {}
fix_hash[ATTRIBUTE_NAME] = error[:correct_attribute_value]
update_document(index_id: error[:index], doc_id: error[:doc], request_body: fix_hash)
puts 'Value corrected to': error[:correct_attribute_value]
puts
end
else
puts 'All values of attribute is ok.'
end