Question

这是我的代码：

 filter {
   json {
     source => "message"
     remove_field => "message"
   } 

   mutate {
     remove_field => "message"
     add_field => ["r","%{[M][result]]autState]}"]
     add_field => ["n", "%{[serviceLogicalId]}"]
     add_field => ["tsc", "%{[T][0][timestamp]}"]
     add_field => ["rtc", "%{[T][0][duration]}"]
     add_field => ["nc", "%{[T][0][transactionName]}"]
     add_field => ["pro", "%{[applicationUUAA]}"]
     add_field => ["con", "%{[aap]}"]
     add_field => ["c", "%{[callingChannel]}"]
     add_field => ["p", "%{[productService]}"]
     add_field => ["es", "services"]
     add_field => ["ci", "%{[cloneId]}"]
     add_field => ["m", "%{[accessMedium]}"]
     add_field => ["s", "%{[productService]}"]
     add_field => ["ui", "%{[contactId]}"] 
     add_field => ["ts", "%{timestamp}"]        
   }

   prune { 
     whitelist_names => ["es", "d", "r", "m", "tsc", "n", "rtc", "nc", "pro", "con", "ci", "ui", "c", "p", "s"] 
   }  
 }  
 output {
   elasticsearch {
   host => "192.168.2.101"
   index => "services"                         
 }

}

我想在whitelist_names中插入字段，但除了这些字段之外还有输出字段（timestamp，contactId，productService，accessMedium，cloneId，productService，callingChannel，aap，applicationUUAA，serviceLogicalId也来自ES）。

仅此字段，即不是数组的字段。

Answer 1

因此，如果我正确理解了此过滤器，您只希望在prune whitelist_names列表中指定的字段进入ES。你想放弃所有其他领域。

好吧也许试试：

 filter {
   json {
     source => "message"
     remove_field => "message"
   }

   mutate {
     add_field => {
       "r" => "%{[M][result]]autState]}",
       "n", "%{[serviceLogicalId]}",
       "tsc", "%{[T][0][timestamp]}",
       "rtc", "%{[T][0][duration]}",
       "nc", "%{[T][0][transactionName]}",
       "pro", "%{[applicationUUAA]}",
       "con", "%{[aap]}",
       "c", "%{[callingChannel]}",
       "p", "%{[productService]}",
       "es", "services",
       "ci", "%{[cloneId]}",
       "m", "%{[accessMedium]}",
       "s", "%{[productService]}",
       "ui", "%{[contactId]}",
       "ts", "%{timestamp}"
     }
   }
   prune {
     whitelist_names => ["es", "d", "r", "m", "tsc", "n", "rtc", "nc", "pro", "con", "ci", "ui", "c", "p", "s"]
     add_tag => [ "pruned" ]
     }
 }

 output {
   elasticsearch {
   host => "192.168.2.101"
   index => "services"
   }
 }

我不确定你为什么要删除remove_field =＆gt; ＆＃34;消息＆＃34;两次，所以我删除了它。

我已经返回了prune方法，并告诉它add_tags，这样你至少知道正在调用prune方法。否则，会有一个小的语法更改，但您可能想要试一试，看看它是否按预期工作。

据我从文档中可以看出，您正在使用剪枝功能，如果它仍然不起作用，您可能最好进入irc频道（freenode.net上的#logstash）与之聊天他们中的一些人可能遇到过这个问题。 GL！

Answer 2

我有解决这个问题的方法：

 filter {
json {
 source => "message"
 remove_field => "message"
 target => "e."

  }

 mutate {    
  add_field => ["r","%{[e.][M][result]]autState]}"]
  add_field => ["n", "%{[e.][serviceLogicalId]}"]
  add_field => ["tsc", "%{[e.][T][0][timestamp]}"]
  add_field => ["rtc", "%{[e.][T][0][duration]}"]
  add_field => ["nc", "%{[e.][T][0][transactionName]}"]
  add_field => ["pro", "%{[e.][applicationUUAA]}"]
  add_field => ["con", "%{[e.][aap]}"]
  add_field => ["c", "%{[e.][callingChannel]}"]
  add_field => ["p", "%{[e.][productService]}"]
  add_field => ["es", "services"]
  add_field => ["ci", "%{[e.][cloneId]}"]
  add_field => ["m", "%{[e.][accessMedium]}"]
  add_field => ["s", "%{[e.][productService]}"]
  add_field => ["ui", "%{[e.][contactId]}"] 
  add_field => ["ts", "%{[e.][timestamp]}"] 

  }

prune {
   whitelist_names => ["es", "d", "r", "m", "tsc", "n", "rtc", "nc", "pro", "con", "ci","ui","c","p", "s"]

 }
}

Answer 3

您的问题的根源是您尝试将whitelist_names传递给字段列表。它实际上需要一个正则表达式列表。

因此，在您的示例中，“d”“r”“m”“n”将匹配包含这些字母的任何字段。

做你想做的事：

prune { 
  whitelist_names => ["^es$", "^d$", "^r$", "^m$", "^tsc$", "^n$", "^rtc$", "^nc$", "^pro$", "^con$", "^ci$", "^ui$", "^c$", "^p$", "^s$"] 
}

logstash中的剪枝过滤器中的错误whitelist_name？

3 个答案: