我一直在努力创建一个syslog应用程序,在该应用程序中,我必须收集使用Fluentd生成的各种日志,并将它们发送给Cassandra,以便稍后进行分析。我必须使用Kafka作为一种缓冲区或流处理器。要在Fluentd上使用Kafka,我必须配置Fluentd。
我已经安装了Fluentd,Cassandra,Kafka和Zookeeper,它们都独立运行,没有任何错误。但是,当我尝试使用fluent.conf
文件将Kafka连接至流利时,我遇到了多个错误。
我的fluent.conf
文件是:
<source>
@type syslog
bind 0.0.0.0
port 5140
tag default
protocol_type udp
source_hostname_key source_host
<parse>
message_format auto
</parse>
</source>
<match **>
@type stdout
</match>
<match app.**>
@type kafka_buffered
zookeeper localhost:2181
zookeeper_path root/attackfence/kafka_2.11-2.2.0/zookeeeper
partition_key partition
partition_key_key partition_key
message_key_key message_key
default_topic nil
default_partition_key nil
default_message_key nil
output_data_type json
output_include_tag false
output_include_time false
exclude_topic_key false
exclude_partition_key false
get_kafka_client_log true
</match>
我得到的错误是:
2019-05-23 18:22:44 +0530 [error]: #0 unexpected error error_class=NoMethodError error="undefined method `each' for nil:NilClass"
2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluent-plugin-kafka-0.9.4/lib/fluent/plugin/out_kafka_buffered.rb:117:in `refresh_client'
2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluent-plugin-kafka-0.9.4/lib/fluent/plugin/out_kafka_buffered.rb:208:in `start'
2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/compat/call_super_mixin.rb:42:in `start'
2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/root_agent.rb:165:in `block in start'
2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/root_agent.rb:154:in `block (2 levels) in lifecycle'
2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/root_agent.rb:153:in `each'
2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/root_agent.rb:153:in `block in lifecycle'
2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/root_agent.rb:140:in `each'
2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/root_agent.rb:140:in `lifecycle'
2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/root_agent.rb:164:in `start'
2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/engine.rb:274:in `start'
2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/engine.rb:219:in `run'
2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/supervisor.rb:799:in `run_engine'
2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/supervisor.rb:549:in `block in run_worker'
2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/supervisor.rb:724:in `main_process'
2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/supervisor.rb:544:in `run_worker'
2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/lib/fluent/command/fluentd.rb:316:in `<top (required)>'
2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/rubies/ruby-2.6.0/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/rubies/ruby-2.6.0/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/gems/fluentd-1.3.3/bin/fluentd:8:in `<top (required)>'
2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/bin/fluentd:23:in `load'
2019-05-23 18:22:44 +0530 [error]: #0 /usr/local/rvm/gems/ruby-2.6.0/bin/fluentd:23:in `<main>'