Logstash 查找弹性搜索索引

时间:2021-04-29 10:13:53

标签: csv elasticsearch logstash lookup

我有这个用例,我想在其中查找 logstash 摄取数据以在 elasticsearch 上建立索引。 索引是testlookup。它有 2 个主要列:id_faculty,faculty.id_faculty 是独一无二的。我想根据 id_faculty 查找 其他数据基于 csv,包含 3 列:"id_student","name_student","id_faculty"

查询模板.json


{
  "size": 1,
  "query":{
    "bool": {
      "should": [
        {
          "match": { "id_faculty": "%{[id_faculty]}" }
        }
      ]
    }
  }
}

logstash.conf

input
{
        beats
        {
                port =>5081
        }
}
filter
{
    csv
    {
        skip_header=> true
        columns=>["id_student","name_student","id_faculty"]
        separator => ","
    }
    elasticsearch
    {
        hosts=> "x.x.x.246"
        query_template=>"query-template.json"
        index=>"testlookup"
        fields =>
        {
            "faculty"=>"[faculty]"
        }
    }
}

output
{
        stdout
        {
                codec => rubydebug
        }

}

但输出从不返回 faculty,它只返回 3 作为 "id_student","name_student","id_faculty"

输出样本

{
        "@timestamp" => 2021-04-29T10:14:59.951Z,
              "host" => {
        "name" => "XXXXXXXXXXXXX"
    },
             "agent" => {
             "version" => "7.7.1",
        "ephemeral_id" => "3278c4a5-4ed2-4e63-8d34-6b05a9a46a4e",
            "hostname" => "XXXXXXXXXXXXX",
                  "id" => "5e2fec03-bbdc-4f91-acc9-4ab36c7268db",
                "type" => "filebeat"
    },
               "log" => {
        "offset" => 119,
          "file" => {
            "path" => "D:\\NGETESLOOKUP\\source\\list.txt"
        }
    },
              "tags" => [
        [0] "beats_input_codec_plain_applied"
    ],
       "id_faculty" => "08",
             "input" => {
        "type" => "log"
    },
          "@version" => "1",
               "ecs" => {
        "version" => "1.5.0"
    },
      "id_student" => "0007",
           "message" => "0007,ggg,08",
    "name_student" => "ggg"
}

我错过了什么吗?

0 个答案:

没有答案