使用Logstah和XPath处理嵌套对象

时间:2017-08-29 12:12:56

标签: elasticsearch xpath logstash

我有一个XML结构

<Event>
  <Core Id="10233" />
  <Parameters>
      <Parameter EngValue="1.0" DecValue="1.0" />
      <Parameter EngValue="GCOM" />
      <Parameter EngValue="1.0" DecValue="1.0" />
  </Parameters>
</Event>

我希望Logstash输出这个结构:

{
  "CoreID" : "10233",
  "Parameter" : [
    {
      "EngValue" : "1.0",
      "DecValue" : "1.0"
    },
    {
      "EngValue" : "GCOM"
    },
    {
      "EngValue" : "1.0",
      "DecValue" : "1.0"
    }
  ]
}

我尝试使用XML过滤器:

xpath => [
    "/Event/Core/@Id", "CoreID",
    "/Event/Parameters/Parameter/@DecValue", "[Parameter][DecValue]",
    "/Event/Parameters/Parameter/@EngValue", "[Parameter][EngValue]",
]

但到目前为止我只得到了这个:

{
  "CoreID" : "10233",
  "Parameter" : {
      "EngValue" : ["1.0", "GCOM", "1.0"],
      "DecValue" : ["1.0", "1.0"]
    }
}

如何将“参数”设为数组而不是其属性?

我尝试使用Ruby过滤器来创建&#34;参数&#34;宾语。但是在XML过滤器之后(即当我的Ruby过滤器开始时)我必须&#34;加入&#34;数组"EngValue" : ["1.0", "GCOM", "1.0"]"DecValue" : ["1.0", "1.0"]

问题是对于“EngValue”的每个元素,我不知道“DecValue”中的相应元素。

1 个答案:

答案 0 :(得分:1)

如果你不介意使用ruby,我会建议这样的事情:

filter {
  xml {
    source => "message"
    target => "parsed"
  }

  ruby {
      code => '
        event.set("Parameter", event.get("[parsed][Parameters][0][Parameter]"))
        event.set("CoreId", event.get("[parsed][Core][0][Id]"))
      '
  }


  mutate {
    remove_field => ["message", "parsed"]
  }
}

应该输出:

{
    "@timestamp" => 2017-08-29T13:45:46.112Z,
      "@version" => "1",
          "host" => "my-host",
     "Parameter" => [
        [0] {
            "DecValue" => "1.0",
            "EngValue" => "1.0"
        },
        [1] {
            "EngValue" => "GCOM"
        },
        [2] {
            "DecValue" => "1.0",
            "EngValue" => "1.0"
        }
    ],
        "CoreId" => "10233"
}