Question

我最近开始学习ELK，但是很难理解如何解析XML数据。我想解析如下所示的XML文件：

<Name nameID="xxxx">
  <Type p="1">xxxxxx</Type>
  <Type p="2">xxxxxx</Type>
    .
    .
  <Type p="9">xxxxx</Type>
  <Value obj="1"> 
    <r p="1">5.94</r>
    <r p="2">62.19</r>
    .
    .
    <r p="9">7.19</r>
  </Value>
  <Value obj="2"> 
    <r p="1">5.94</r>
    <r p="2">62.19</r>
    .
    .
    <r p="9">7.19</r>
  </Value>
</Name>
<Name nameID="yyyy">
  <Type p="1">yyyyy</Type>
  <Type p="2">yyyyyy</Type>
  <Type p="3">yyyy</Type>
  <Value obj="1"> 
    <r p="1">54.94</r>
    <r p="2">6.19</r>
    <r p="3">0</r>
  </Value>
</Name>

我想得到类似的东西：在输出中

"NameID = name1
Type = Type1
obj = obj1
Value = xx
"
"NameID = name1
Type = Type2
obj = obj1
Value = xx
"
"NameID = name1
Type = Type3
obj = obj1
Value = xx
"
...etc
and then
"NameID = name1
Type = Type1
obj = obj2
Value = xx
"
"NameID = name1
Type = Type2
obj = obj2
Value = xx
"
....etc

我使用了这个logstash.conf，但没有得到真正需要的东西（每个字段都有一个数组）

input {
    file {
        path => "/home/test/data.xml"
        start_position => beginning
        sincedb_path => "/dev/null"
        codec => multiline
        {
            pattern => "<Name"
            negate => true
            what => "previous"
        }
    }
}
filter
{
    xml {
        source => "message"
        target => "parsed"
        add_tag => "xml"
        xpath => [
            "//Name/@nameID","Name",
            "//Type/@p","TypeID",
            "//Type/text()","Type",
            "//Value/@obj","Obj",
            "//r/text()","value"]

Answer 1

使用日志记录ruby filter plugin。（需要代码中的require宝石吗？）
使用ruby parse XML，以
构建可用于索引的json文档

Answer 2

解决方案：

filter { xml { source => "message" store_xml => true target => "theXML" force_array => false } }
split { field => "[theXML][Type]" }
split { field => "[theXML][Value]" }
split { field => "[theXML][Value][r]" }

，然后在输出中使用：

output{
if [theXML][Type][p]==[theXML][Value][p]{
elasticsearch ....}}

希望可以帮助某人；）

使用ELK解析复杂的XML文件

2 个答案: