Logstash

Question

我已将JSON作为Logstash管道的输入：

{
  "action": "UPLOAD",
  "who": "123",
  "when": "2016 Jun 14 12:00:12",
  "data": {
    "doc_id": "2345",
    "doc_name": "xyz.pdf"
  },
  "header": {
    "proj_id": "P123",
    "logtype": "userlogs"
  },
  "comments": "Check comments"
}

我想执行以下操作：

1）解析这个JSON - 这样我就有了一个新的字段“user”，其值为静态字符串“User-”和JSON中的“who”字段的串联。例如 - “User-123”

2）仅存储ES中的相关字段 - 例如ElasticSearch中的action，who，when，header.proj_id，header.logtype。并留下其余的字段而不存储它们。

我尝试使用以下配置，但目前它将我的JSON的所有字段存储到弹性搜索中。

input {
    rabbitmq {
        type => "businesslogs"
        host => "localhost"
        exchange => "auditexchange"
        exchange_type => "fanout"
        queue => "auditqueue"
        auto_delete => false
        durable => true
        ack => true
        codec => json
    }
}

output {
    if [type] == "businesslogs" {
        elasticsearch {
            hosts => ["localhost:9200"]
            index => "businesslogs"
            document_type => "%{action}"
        }
    }
}

Answer 1

您只需添加mutate过滤条件即可在活动中添加/删除某些字段：

filter {
    mutate {
        add_field => {
            "user" => "User-%{who}"
        }
        remove_field => ["data", "comments"]
    }
}