经过几天的运行后,Kafka LeaderNotAvailableException

时间:2015-10-23 09:03:54

标签: linux scala apache-kafka apache-zookeeper

我在Java 8上有一个带有Zookeeper 3.4.6的Kafka 0.8.2.1的环境,并且在Linux(Centos7)环境中运行良好。经过几天(大约1个月),Kafka不再使用消息发布,包括来自控制台消费者,导致以下例外情况。

 [2015-10-23 10:49:25,016] WARN Error while fetching metadata [{TopicMetadata for topic talBI -> No partition metadata for topic talBI due to kafka.common.LeaderNotAvailableException}] for topic [talBI]: class kafka.common.LeaderNotAvailableException  (kafka.producer.BrokerPartitionInfo) [2015-10-23 10:49:25,026] WARN Error while fetching metadata [{TopicMetadata for topic talBI -> No partition metadata for topic talBI due to kafka.common.LeaderNotAvailableException}] for topic [talBI]: class kafka.common.LeaderNotAvailableException  (kafka.producer.BrokerPartitionInfo) [2015-10-23 10:49:25,026] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: talBI (kafka.producer.async.DefaultEventHandler) [2015-10-23 10:49:25,138] WARN Error while fetching metadata [{TopicMetadata for topic talBI -> No partition metadata for topic talBI due to kafka.common.LeaderNotAvailableException}] for topic [talBI]: class kafka.common.LeaderNotAvailableException  (kafka.producer.BrokerPartitionInfo) [2015-10-23 10:49:25,146] WARN Error while fetching metadata [{TopicMetadata for topic talBI -> No partition metadata for topic talBI due to kafka.common.LeaderNotAvailableException}] for topic [talBI]: class kafka.common.LeaderNotAvailableException  (kafka.producer.BrokerPartitionInfo) [2015-10-23 10:49:25,147] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: talBI (kafka.producer.async.DefaultEventHandler) [2015-10-23 10:49:25,256] WARN Error while fetching metadata [{TopicMetadata for topic talBI -> No partition metadata for topic talBI due to kafka.common.LeaderNotAvailableException}] for topic [talBI]: class kafka.common.LeaderNotAvailableException  (kafka.producer.BrokerPartitionInfo) [2015-10-23 10:49:25,265] WARN Error while fetching metadata [{TopicMetadata for topic talBI -> No partition metadata for topic talBI due to kafka.common.LeaderNotAvailableException}] for topic [talBI]: class kafka.common.LeaderNotAvailableException  (kafka.producer.BrokerPartitionInfo) [2015-10-23 10:49:25,265] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: talBI (kafka.producer.async.DefaultEventHandler) [2015-10-23 10:49:25,377] WARN Error while fetching metadata [{TopicMetadata for topic talBI -> No partition metadata for topic talBI due to kafka.common.LeaderNotAvailableException}] for topic [talBI]: class kafka.common.LeaderNotAvailableException  (kafka.producer.BrokerPartitionInfo) [2015-10-23 10:49:25,390] WARN Error while fetching metadata [{TopicMetadata for topic talBI -> No partition metadata for topic talBI due to kafka.common.LeaderNotAvailableException}] for topic [talBI]: class kafka.common.LeaderNotAvailableException  (kafka.producer.BrokerPartitionInfo) [2015-10-23 10:49:25,390] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: talBI (kafka.producer.async.DefaultEventHandler) [2015-10-23 10:49:25,500] WARN Error while fetching metadata [{TopicMetadata for topic talBI -> No partition metadata for topic talBI due to kafka.common.LeaderNotAvailableException}] for topic [talBI]: class kafka.common.LeaderNotAvailableException  (kafka.producer.BrokerPartitionInfo) [2015-10-23 10:49:25,501] ERROR Failed to send requests for topics talBI with correlation ids in [0,8] (kafka.producer.async.DefaultEventHandler) [2015-10-23 10:49:25,502] ERROR Error in handling batch of 1 events (kafka.producer.async.ProducerSendThread) kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries.
        at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
        at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
        at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
        at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
        at scala.collection.immutable.Stream.foreach(Stream.scala:547)
        at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
        at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)

关于该主题的描述导致以下内容并且似乎正常:

Topic:talBI     PartitionCount:1        ReplicationFactor:1     Configs:
    Topic: talBI    Partition: 0    Leader: 1       Replicas: 1     Isr: 1

我尝试使用以下命令运行kafka-preferred replica-election命令行工具,但它仍然会出现同样的问题。

kafka-preferred-replica-election --zookeeper md1qacat01.lnx.ix.com:2181/kafka

由于我们正在评估使用Kafka进行生产,因此这仍然在质量保证环境中。我是否缺少某些配置或可能发生的事情的任何想法?

1 个答案:

答案 0 :(得分:1)

当您的网络设置发生变化时,我看到了这种行为。
您可以尝试在KAFKA server.properties 中设置 advertised.host.name =“kafka服务器主机名”,然后重试。