使用Logstash使用不同的HDFS输出的多个Kafka主题不起作用

时间:2018-07-30 07:57:04

标签: apache-kafka logstash

我使用logstash来消耗kafka数据并将日志保存到hdfs,并且我希望使用不同的kafka主题拆分hdfs输出,我的logstash配置如下:

input {
  kafka {
    bootstrap_servers => "192.168.1.14:9092"
    group_id => "logstash_act"
    topics => ["act", "log"]
    decorate_events => true
    consumer_threads => 5
    }
}
output {
    if [kafka][topic] == "act" {
        webhdfs {
            host => "192.168.1.8"
            port => 50070
            user => "root"
            path => "/ubuntu/input/logstash-%{+YYYY}-%{+MM}-%{+dd}.act"
            codec => plain { format => "%{message}" }
            idle_flush_time => 10
            retry_interval => 0.5 flush_size => 500
        }
    }

    else if [kafka][topic] == "log" {
        webhdfs {
            host => "192.168.1.8"
            port => 50070
            user => "root"
            path => "/ubuntu/input/logstash-%{+YYYY}-%{+MM}-%{+dd}.log"
            codec => plain { format => "%{message}" }
            idle_flush_time => 10
            retry_interval => 0.5 flush_size => 500
        }
    }
}

没有错误,但是根本没有输出。

0 个答案:

没有答案