我有一个python脚本,它将JSON对象(逐行)写入/var/log/myLog.json,格式如下:
{"timestamp":"2016-07-21T01:20:04.392799-0400","in_iface":"docker0","event_type":"alert","src_ip":"172.17.0.2","dest_ip":"172.17.0.3","proto":"ICMP","icmp_type":0,"icmp_code":0,"alert":{"action":"allowed","**gid**":2,"signature_id":2,"rev":0,"signature":"ICMP msg","category":"","severity":3},"payload":"hFuQVwAA","payload_printable":"kk"}
我想使用Logstash以便:
从/var/log/myLog.json
解析gid并转发到另一台机器作为udp消息(给定一个特定的IP地址+端口) - 例如:如果gid == 2然后将此json对象转发到172.123.10.3:10001
此外,我希望能够动态更新Logstash配置文件过滤器(Aka,以便能够添加另一个规则,如:“如果gid == x然后将此json对象转发给另一个IP)。
我该怎么办?
Logstash配置文件应该如何? 插入/删除动态过滤器的命令如何?
谢谢,伙计们。
答案 0 :(得分:0)
您可以按以下配置运行logstash。 我测试了两个样本json数据。
{"timestamp":"2016-07-21T01:20:04.392799-0400","in_iface":"docker0","event_type":"alert","src_ip":"172.17.0.2","dest_ip":"172.17.0.3","proto":"ICMP","icmp_type":0,"icmp_code":0,"alert":{"action":"allowed","gid":2,"signature_id":2,"rev":0,"signature":"ICMP msg","category":"","severity":3},"payload":"hFuQVwAA","payload_printable":"kk"}
{"timestamp":"2016-07-21T01:20:04.392799-0400","in_iface":"docker0","event_type":"alert","src_ip":"172.17.0.2","dest_ip":"172.17.0.3","proto":"ICMP","icmp_type":0,"icmp_code":0,"alert":{"action":"allowed","gid":3,"signature_id":2,"rev":0,"signature":"ICMP msg","category":"","severity":3},"payload":"hFuQVwAA","payload_printable":"kk"}
input {
file {
path => "/etc/logstash/jsonSample.log"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
json {
source => "message"
target => "doc"
add_field => {"alert.gid" => "%{[doc][alert][gid]}"}
add_tag => ["tagName_%{[doc][alert][gid]}"]
}
}
output {
if "tagName_2" in [tags] {
stdout {codec => rubydebug}
}else if "tagName_3" in [tags] {
}
}
然后你可以看到结果
{
"message" => "{\"timestamp\":\"2016-07-21T01:20:04.392799-0400\",\"in_iface\":\"docker0\",\"event_type\":\"alert\",\"src_ip\":\"172.17.0.2\",\"dest_ip\":\"172.17.0.3\",\"proto\":\"ICMP\",\"icmp_type\":0,\"icmp_code\":0,\"alert\":{\"action\":\"allowed\",\"gid\":2,\"signature_id\":2,\"rev\":0,\"signature\":\"ICMP msg\",\"category\":\"\",\"severity\":3},\"payload\":\"hFuQVwAA\",\"payload_printable\":\"kk\"}",
"@version" => "1",
"@timestamp" => "2016-07-25T04:41:11.980Z",
"path" => "/etc/logstash/jsonSample.log",
"host" => "baklava",
"doc" => {
"timestamp" => "2016-07-21T01:20:04.392799-0400",
"in_iface" => "docker0",
"event_type" => "alert",
"src_ip" => "172.17.0.2",
"dest_ip" => "172.17.0.3",
"proto" => "ICMP",
"icmp_type" => 0,
"icmp_code" => 0,
"alert" => {
"action" => "allowed",
"gid" => 2,
"signature_id" => 2,
"rev" => 0,
"signature" => "ICMP msg",
"category" => "",
"severity" => 3
},
"payload" => "hFuQVwAA",
"payload_printable" => "kk"
},
"alert.gid" => 2,
"tags" => [
[0] "tagName_2"
]
}
您还可以更改上面应用的配置。
问候。
您可以参考event和json过滤器的配置 https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html
https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html