我有一个Kafka集群,其中包含3个代理配置如下:
Broker-0 : broker.id=0 ; listeners=PLAINTEXT://localhost:9092 ; log.dirs=/opt/data/kafka/logs-0
Broker-1 : broker.id=1 ; listeners=PLAINTEXT://localhost:9093 ; log.dirs=/opt/data/kafka/logs-1
Broker-2 : broker.id=2 ; listeners=PLAINTEXT://localhost:9094 ; log.dirs=/opt/data/kafka/logs-2
制作人将Broker-2定位为发布消息
sh-3.2# ./kafka-console-producer.sh --broker-list localhost:9094 --topic clusterTopic
我开始测试所有经纪人的运行情况
sh-3.2# ./bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic clusterTopic
Topic:clusterTopic PartitionCount:1 ReplicationFactor:3 Configs:
Topic: clusterTopic Partition: 0 Leader: 0 Replicas: 0,1,2 Isr: 2,0,1
sh-3.2#
在这一点上,生产者发布的所有消息都由消费者接收
然后我杀了Broker-0(现任领导人)
sh-3.2# ./bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic clusterTopic
Topic:clusterTopic PartitionCount:1 ReplicationFactor:3 Configs:
Topic: clusterTopic Partition: 0 Leader: 1 Replicas: 0,1,2 Isr: 2,1
sh-3.2#
此时,消费者不再收到生产者发布的消息
然后,我启用了Broker-0
sh-3.2# ./bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic clusterTopic
Topic:clusterTopic PartitionCount:1 ReplicationFactor:3 Configs:
Topic: clusterTopic Partition: 0 Leader: 1 Replicas: 0,1,2 Isr: 2,1,0
sh-3.2#
此时,消费者正确接收生产者发布的所有消息
然后我杀了Broker-1(现任领导人)
sh-3.2# ./bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic clusterTopic
Topic:clusterTopic PartitionCount:1 ReplicationFactor:3 Configs:
Topic: clusterTopic Partition: 0 Leader: 0 Replicas: 0,1,2 Isr: 2,0
sh-3.2#
此时,消费者仍然可以正确接收生产者发布的所有消息。
为什么行为与经纪人的行为如此不同?
更新#1
我发现了一些可能有用的东西:
当我杀死Broker-1时,我在消费者控制台上看到以下警告(很多次)
[2018-06-07 13:59:00,866] WARN [Consumer clientId=consumer-1, groupId=console-consumer-20882] Connection to node 1 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
但是当我杀死Broker-0时,我会看到这些警告信息(很多次)
[2018-06-07 14:04:08,433] WARN [Consumer clientId=consumer-1, groupId=console-consumer-20882] Connection to node 2147483647 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2018-06-07 14:04:08,434] WARN [Consumer clientId=consumer-1, groupId=console-consumer-20882] Connection to node 0 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
我想知道ID = 2147483647的经纪人是什么?