Logstash - 转换xml数据并加载到ES中

时间:2018-04-02 19:08:44

标签: elasticsearch logstash logstash-grok elasticsearch-5 logstash-file

XML格式数据

 <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<details>
    <id>544538</id>
    <name>john</icUrl>
    <version>2.0.0-10</version>

    <OptionalValues>
        <metaData>
            <name>Severity</name>
            <value>Medium</value>
        </metaData>
        <metaData>
            <name>Prioriry</name>
            <value>5</value>
        </metaData>
     <OptionalValues>

    <issue>
        <summary>summary value</summary>
        <description>It is possible to misconfigure an EtherChannel and create a spanning-tree loop. This misconfiguration would potentially overwhelm the switch process. Cisco IOS System Software includes a feature called 'spanning-tree etherchannel guard misconfig' to prevent this issue.</description>
    </issue>
 </details>

使用logstash,我需要像这样转换。 id,名称,版本,严重性,优先级,摘要,描述..如何将xml转换为json输出。 我试图在应用所有字段之前解析id字段。

input {  
    file {
        path => "C:/Users/Desktop/mydata.xml"
            start_position => "beginning"
            sincedb_path => "/dev/null"
    }
}

filter {
    xml {
        source => "message"
        force_array => false
        remove_namespaces =>true   
        store_xml => true
        target => "doc"
        xpath => [  

            "details/id", "myid"
            ]
    }
}

output 
{
    stdout{
        codec => line {     
            format => "%{[myid]}"
        }
    }
}

执行时我在控制台中获取空的myid值。格式化后告诉我进入json如何重定向到es索引。

1 个答案:

答案 0 :(得分:0)

基于此

  

https://www.w3schools.com/xml/xml_xpath.asp

在我看来,你的xpath在开头缺少“/”。它未经过测试,因此无法保证 这显示了es输出的最基本配置:

  

https://logz.io/blog/logstash-tutorial/

如果您需要更详细的选项,我建议使用elasticsearch输出插件的logstash页面。