Logstash匹配多个值

Question

1）这是我的 logstash.conf 文件

ImageView

2）如下面所示，我使用调试器测试了我的grok

3）这是logstash写入elasticsearch的内容

input {
  beats {
    type => beats
    port => 5044
  }
}

filter {
  grok {
    match => { "message" => "\[(?<logtime>([0-9]|[\-\+\.\:\ ])*)\] \[(?<level>([a-z-A-Z])*)\] \[(?<msg>(.)+)\] (?<exception>(.)+)" }
  }
  mutate {
    add_field => [ "logtime", "level", "msg", "exception" ]
    remove_field => [ "beat", "offset", "source", "prospector", "host", "tags" ]
  }  
}

output {
  if [type] == "beats"{
    elasticsearch {
      hosts => "localhost:9200"
      manage_template => false
      index => "%{+YYYY.MM.dd}-container.api" 
      document_type => "%{[@metadata][type]}" 
      user => "elastic"
      password => "secret"
    }
  }
}

4）我想看的是

  {
    "_index": "2019.01.28-container.api",
    "_type": "doc",
    "_id": "pZctlWgBojxJzDZGWqZz",
    "_score": 1,
    "_source": {
      "type": "beats",
      "level": "Debug",
      "@timestamp": "2019-01-28T15:56:41.295Z",
      "msg": [
        "Hosting starting",
        "exception"
      ],
      "@version": "1",
      "logtime": [
        "2019-01-28 15:23:12.911 +03:00",
        "level"
      ],
      "message": "[2019-01-28 15:23:12.911 +03:00] [Debug] [Hosting starting] exception 2",
      "exception": "exception 2",
      "input": {
        "type": "log"
      }
    }
  }

Answer 1

问题出在

mutate {
    add_field => [ "logtime", "level", "msg", "exception" ]
}

您要添加的字段已经由grok过滤器创建，再次这样做是没有用的，因为mutate.addField使用，它只会转换数组中已经存在的字段并将新值添加到数组中散列，它将在字段logtime中添加值level，在字段msg中添加值exception。

Answer 2

mutate {
    add_field => [ "logtime", "level", "msg", "exception" ]
}

这与：

mutate {
        add_field => { 
              "logtime" => "level" 
              "msg" => "exception" 
        }
   }

这就是数组存在且具有多个值的原因。由于您在grok模式中定义了变量名称，因此无需再次指定。因此，如baudsp所说，您可以删除此“添加字段”。