Question

I am absolutely new to Logstash and I am trying to parse my multiline logentries, that are in the following format

<log level="INFO" time="Wed May 03 08:25:03 CEST 2017" timel="1493792703368" host="host"> <msg><![CDATA[Method=GET URL=http://localhost (Vers=[Version], Param1=[param1], Param2=[param1]) Result(Content-Length=[22222], Content-Type=[text/xml; charset=utf-8]) Status=200 Times=TISP:1098/CSI:-/Me:1/Total:1099]]> </msg> </log>

Do you know how to implement the filter in logstash config to be able to index the following fields in elasticsearch

time, host, Vers, Param1, Param2, TISP

Thank you very much

Answer 1

好的，我发现了怎么做。这是我的pipeline.conf文件，它可以正常工作

input {
        beats {
                port => 5044
        }
}

filter {
        xml {
                store_xml => false
                source => "message"
                xpath => [
                 "/log/@level", "level",
                 "/log/@time", "time",
                 "/log/@timel", "unixtime",
                 "/log/@host", "host_org",
                 "/log/@msg", "msg",
                 "/log/msg/text()","msg_txt"
                ]
        }

        grok {
                break_on_match => false
                match => ["msg_txt", "Param1=\[(?<param1>-?\w+)\]"]
                match => ["msg_txt", "Param2=\[(?<param2>-?\w+)\]"]
                match => ["msg_txt", "Vers=\[(?<vers>-?\d+\.\d+)\]"]
                match => ["msg_txt", "TISP:(?<tisp>-?\d+)"]
                match => [unixtime, "(?<customTime>-?\d+)"]
        }
        if "_grokparsefailure" in [tags] {
                drop { }
        }

        mutate {
                convert => { "tisp" => "integer" }
        }

        date {
                match => [ "customTime", "UNIX_MS"]
                target => "@timestamp"
        }
        if "_dateparsefailure" in [tags] {
                drop { }
        }



}

output {
        elasticsearch {
                hosts => "elasticsearch:9200"
                user => user
                password => passwd
        }
}

Logstash Multiline Logfile XML Parsing Filter

1 个答案: