Question

我在我的logstash管道中使用了elasticsearch过滤器。我正确地找到了结果：

filter{
  if [class] == "DPAPIINTERNAL" {
    elasticsearch {
      hosts => "10.1.10.16"
      index => "dp_audit-2017.02.16"
      query_template => "/home/vittorio/Documents/elastic-queries/matching-requestaw.json"
    }
  }
}

正如你所看到的，我正在使用＆＃34; query_template＆＃34;这是：

{
    "query": {
      "query_string": {
       "query": "class:DPAPI AND request.aw:%{[aw]}"
      }
    },
   "_source": ["end_point", "vittorio"]
 }

告诉elastichsearch使用匹配＆＃34; aw＆＃34;的特定类查找日志。使用DPAPIINTERNAL日志。

完美！但是现在我找到了结果，我想从中添加一些字段并将它们附加到我的DPAPIINTERNAL日志中，例如，我想采取＆＃34; end_point＆＃34;并将其添加到新密钥＆＃34; vittorio＆＃34;在我的日志里面。

这种情况没有发生，我也不明白为什么。

这是我正在使用查询查看的日志：

{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 1,
    "hits": [
      {
        "_index": "dp_audit-2017.02.16",
        "_type": "logs",
        "_id": "AVpHoPHPuEPlW12Qu",
        "_score": 1,
        "_source": {
          "svc": "dp-1.1",
          "request": {
            "method": "POST|PATCH|DELETE",
            "aw": "prova",
            "end_point": "/bank/6311",
            "app_instance": "7D1-D233-87E1-913"
          },
          "path": "/home/vittorio/Documents/dpapi1.json",
          "@timestamp": "2017-02-16T15:53:33.214Z",
          "@version": "1",
          "host": "Vito",
          "event": "bank.add",
          "class": "DPAPI",
          "ts": "2017-01-16T19:20:30.125+01:00"
        }
      }
    ]
  }
}

Answer 1

您需要在elasticsearch过滤器中指定fields parameter，如下所示：

elasticsearch {
  hosts => "10.1.10.16"
  index => "dp_audit-2017.02.16"
  query_template => "/home/vittorio/Documents/elastic-queries/matching-requestaw.json"
  fields => { "[request][end_point]" => "vittorio" }
}

请注意，由于end_point是嵌套字段，因此您需要修改查询模板中的_source，如下所示：

"_source": ["request.end_point"]

Answer 2

问题只是您不必使用query_template指定“new”字段。

"_source": ["request"] # here you specify the field you want from the query result.

然后

filter{
  if [class] == "DPAPIINTERNAL" {
    elasticsearch {
      hosts => "10.1.10.16"
      index => "dp_audit-2017.02.16"
      query_template => "/home/vittorio/Documents/elastic-queries/matching-requestaw.json"
      fields => {"request" => "new_key"} # here you add the fields and will tell elastich filter to put request inside new_key
    }
  }
}

这对我有用！

在logstash管道中使用elasticsearch过滤器

2 个答案: