Question

在logstash 5.0版中，数据遗漏了很多，它是一个严重的错误，当配置文件配置这么多次，它没用，数据丢失再次发生和agin，如何使用logstash来收集日志事件属性？

任何回复都会感谢

Answer 1

Logstash就是从特定位置读取日志，并根据您感兴趣的信息，您可以在弹性搜索或其他输出中创建索引。 logstash conf的示例

input { 
file {
# PLEASE SET APPROPRIATE PATH WHERE LOG FILE AVAILABLE
        #type => "java"
        type => "json-log"
        path => "d:/vox/logs/logs/vox.json"
        start_position => "beginning"
          codec => json
  }
 }

filter {
 if [type] == "json-log" {
   grok {
       match => { "message" => "UserName:%{JAVALOGMESSAGE:UserName} -DL_JobID:%{JAVALOGMESSAGE:DL_JobID} -DL_EntityID:%{JAVALOGMESSAGE:DL_EntityID} -BatchesPerJob:%{JAVALOGMESSAGE:BatchesPerJob} -RecordsInInputFile:%{JAVALOGMESSAGE:RecordsInInputFile} -TimeTakenToProcess:%{JAVALOGMESSAGE:TimeTakenToProcess} -DocsUpdatedInSOLR:%{JAVALOGMESSAGE:DocsUpdatedInSOLR} -Failed:%{JAVALOGMESSAGE:Failed} -RecordsSavedInDSE:%{JAVALOGMESSAGE:RecordsSavedInDSE} -FileLoadStartTime:%{JAVALOGMESSAGE:FileLoadStartTime} -FileLoadEndTime:%{JAVALOGMESSAGE:FileLoadEndTime}" }
       add_field => ["STATS_TYPE", "FILE_LOADED"]
    }

}
}
filter {

  mutate {
  # here converting data type

        convert => { "FileLoadStartTime" => "integer" }
        convert => { "RecordsInInputFile" => "integer" }




  }
}

output {
elasticsearch { 
# PLEASE CONFIGURE ES IP AND PORT WHERE LOG DOCs HAS TO PUSH

document_type => "json-log"
hosts => ["localhost:9200"]
        # action => "index"       
        # host => "localhost"
         index => "locallogstashdx_new"
        # workers => 1
 }
  stdout { codec => rubydebug }
  #stdout { debug => true }
}

要了解更多，你可以去许多可用的网站，如 https://www.elastic.co/guide/en/logstash/current/first-event.html

Logstash中遗漏的数据？

1 个答案: