如何在kafka 0.80中调试领导者问题?

时间:2016-01-20 05:28:41

标签: apache-kafka apache-zookeeper

我一直试图用小型kafka实例来隔离问题。 (必须使用kafka,因为我不准备重新实现的其他软件取决于它。)这件事只需要运行几周,但它甚至没有这样做。

具体来说,这是我目前的问题:

$ bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic
 test
Created topic "test".
$ bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
This is a message
This is another message[2016-01-20 05:13:56,290] WARN Error while fetching metadata [{TopicMetadata for topic test -> 
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNot
AvailableException  (kafka.producer.BrokerPartitionInfo)
[2016-01-20 05:13:56,296] WARN Error while fetching metadata [{TopicMetadata for topic test -> 
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNot
AvailableException  (kafka.producer.BrokerPartitionInfo)
[2016-01-20 05:13:56,297] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: test
 (kafka.producer.async.DefaultEventHandler)
[2016-01-20 05:13:56,401] WARN Error while fetching metadata [{TopicMetadata for topic test -> 
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNot
AvailableException  (kafka.producer.BrokerPartitionInfo)
[2016-01-20 05:13:56,404] WARN Error while fetching metadata [{TopicMetadata for topic test -> 
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNot
AvailableException  (kafka.producer.BrokerPartitionInfo)
[2016-01-20 05:13:56,404] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: test
 (kafka.producer.async.DefaultEventHandler)
[2016-01-20 05:13:56,507] WARN Error while fetching metadata [{TopicMetadata for topic test -> 
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNot
AvailableException  (kafka.producer.BrokerPartitionInfo)
[2016-01-20 05:13:56,511] WARN Error while fetching metadata [{TopicMetadata for topic test -> 
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNot
AvailableException  (kafka.producer.BrokerPartitionInfo)
[2016-01-20 05:13:56,511] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: test
 (kafka.producer.async.DefaultEventHandler)
[2016-01-20 05:13:56,614] WARN Error while fetching metadata [{TopicMetadata for topic test -> 
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNot
AvailableException  (kafka.producer.BrokerPartitionInfo)
[2016-01-20 05:13:56,618] WARN Error while fetching metadata [{TopicMetadata for topic test -> 
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNot
AvailableException  (kafka.producer.BrokerPartitionInfo)
[2016-01-20 05:13:56,618] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: test
 (kafka.producer.async.DefaultEventHandler)
[2016-01-20 05:13:56,721] WARN Error while fetching metadata [{TopicMetadata for topic test -> 
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNot
AvailableException  (kafka.producer.BrokerPartitionInfo)
[2016-01-20 05:13:56,723] ERROR Failed to send requests for topics test with correlation ids in [0,8] (kafka.producer.async.DefaultEv
entHandler)
[2016-01-20 05:13:56,724] ERROR Error in handling batch of 1 events (kafka.producer.async.ProducerSendThread)
kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries.
        at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
        at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:104)
        at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:87)
        at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:67)
        at scala.collection.immutable.Stream.foreach(Stream.scala:254)
        at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:66)
        at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:44)
...

换句话说,我正在尝试使用文档中的简单测试用例,并且这样做不正确 - 在新创建的主题上,kafka已经存在元数据问题。

查看kafka日志,最近更新的唯一一个是server.log,它只有关于关闭套接字的信息消息。

看看zookeeper日志,我确实看到了:

2016-01-20 05:13:16,028 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:57826
2016-01-20 05:13:16,031 [myid:] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@822] - Connection request from old client /127.0.0.1:57826; will be dropped if server is in r-o mode
2016-01-20 05:13:16,031 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@868] - Client attempting to establish new session at /127.0.0.1:57826
2016-01-20 05:13:16,032 [myid:] - INFO  [SyncThread:0:ZooKeeperServer@617] - Established session 0x1525d2609e90006 with negotiated timeout 30000 for client /127.0.0.1:57826
2016-01-20 05:13:16,164 [myid:] - INFO  [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x1525d2609e90006 type:setData cxid:0x3 zxid:0x452 txntype:-1 reqpath:n/a Error Path:/config/topics/test Error:KeeperErrorCode = NoNode for /config/topics/test
2016-01-20 05:13:16,179 [myid:] - INFO  [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x1525d2609e90006 type:create cxid:0x4 zxid:0x453 txntype:-1 reqpath:n/a Error Path:/config/topics Error:KeeperErrorCode = NodeExists for /config/topics
2016-01-20 05:13:16,190 [myid:] - INFO  [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@494] - Processed session termination for sessionid: 0x1525d2609e90006
2016-01-20 05:13:16,192 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1007] - Closed socket connection for client /127.0.0.1:57826 which had sessionid 0x1525d2609e90006

但是,如果我理解2011年的消息,我看到了这种错误,这应该是正常的,并自动处理。

因此...

我应该做些什么来隔离和解决领导者问题的缺失? (另请注意,我正在尝试此测试的原因是因为我的生产主题是(a)没有将事件写入磁盘和(b)当我尝试向其编写测试消息时,给我相同类型的无领导错误。)< / p>

0 个答案:

没有答案