如何在Logstash中合并两个事件?

时间:2018-12-05 11:32:03

标签: elasticsearch logstash logstash-grok elasticsearch-aggregation

我正在尝试通过logstash将日志文件解析为elasticsearch。

我想将以下日志作为单个事件(即作为单个文档)发送到elasticsearch。

这是我的日志文件,如下所示:

######################################################
ETL Wrapper Initializing - 09/27/2018 06:33:57
######################################################
------------------------------------------------------
Wrapper Information - 09/27/2018 06:33:57
------------------------------------------------------

------------------------------------------------------
Reading Component Log Port Files - 09/27/2018 06:34:53
------------------------------------------------------
-- >     -- > Found 3 files and only merge non-zero byte files
------------------------------------------------------
Renaming Reject Files - 09/27/2018 06:34:56
------------------------------------------------------
######################################################
Sending Notifications - 09/27/2018 06:34:56
######################################################
------------------------------------------------------
Setting Exit Status - 09/27/2018 06:34:56
------------------------------------------------------
######################################################
ETL Wrapper Finalizing - 09/27/2018 06:34:56
######################################################
------------------------------------------------------

这是我的logstash配置:

input {
file {
path => "D:/logs/file.log"
start_position => "beginning"
}
}
filter{
grok {
match => {"message" => "ETL Wrapper Initializing - %{DATESTAMP:JobStartTime}"}
match => {"message" => "ETL Wrapper Finalizing - %{DATESTAMP:JobEndTime}"}
}
if "_grokparsefailure" in [tags]{
drop{}
}
if [message] =~ /^$/ {
        drop { }
}
mutate{
    remove_field => ["@version","host","message" ]
    }
}
output {
elasticsearch {
    hosts => "http://localhost:9200/"
    index => "success_index"
    }
stdout { codec => rubydebug }
}

以上配置的输出:

{
    "JobStartTime" => "09/27/2018 09:33:41",
      "@timestamp" => 2018-12-05T10:55:44.698Z,
            "path" => "D:/logs/file.log"
}
{
    "JobEndTime" => "09/27/2018 09:34:16",
    "@timestamp" => 2018-12-05T10:55:44.784Z,
          "path" => "D:/logs/file.log"
}

我的预期输出:

{
    "JobStartTime" => "09/27/2018 09:33:41",
      "@timestamp" => 2018-12-05T10:55:44.698Z,
            "path" => "D:/logs/file.log"
    "JobEndTime" => "09/27/2018 09:34:16"
}

如何将“ JobStartTime”和“ JobEndTime”合并到单个文档中?

任何帮助都是有意义的。 预先感谢!

0 个答案:

没有答案