验证分区的Kafka重新分配

时间:2019-01-24 17:13:15

标签: apache-kafka

我想验证主题的平衡,包括副本和Isr

我将给出一些关于平衡错误的示例(master01是zookeper服务器,我们定义了3个副本)

在该示例中,我们可以看到某些Isr缺少kafka经纪人

/usr/hdp/current/kafka-broker/bin/kafka-topics.sh  --describe --zookeeper  master01:2181 --topic mno.de.pola.trump  Topic:mno.de.pola.trump  PartitionCount:100  ReplicationFactor:3  Configs:  Topic:
mno.de.pola.trump  Partition: 0  Leader: 1017  Replicas: 1017,1018,1016   Isr: 1017,1018,1016  Topic:
mno.de.pola.trump  Partition: 1  Leader: 1018  Replicas: 1018,1016,1017   Isr: 1018,1017,1016  Topic:
mno.de.pola.trump  Partition: 2  Leader: 1016  Replicas: 1016,1017,1018   Isr: 1016,1018  Topic:
mno.de.pola.trump  Partition: 3  Leader: 1017  Replicas: 1017,1016,1018   Isr: 1017,1018,1016  Topic:
mno.de.pola.trump  Partition: 4  Leader: 1018  Replicas: 1018,1017,1016   Isr: 1018,1017,1016  Topic:
mno.de.pola.trump  Partition: 5  Leader: 1016  Replicas: 1016,1018,1017   Isr: 1016,1018,1017  Topic:
mno.de.pola.trump  Partition: 6  Leader: 1017  Replicas: 1017,1018,1016   Isr: 1017,1018

在以下示例中,我们可以看到分区10上的Leader更多地位于代理1017上

/usr/hdp/current/kafka-broker/bin/kafka-topics.sh  --describe --zookeeper  master01:2181 --topic mno.de.pola.trump  Topic:mno.de.pola.trump   PartitionCount:100  ReplicationFactor:3  Configs:  Topic:
mno.de.pola.trump  Partition: 0  Leader: 1017  Replicas: 1017,1018,1016  Isr: 1017,1018,1016  Topic:
mno.de.pola.trump  Partition: 1  Leader: 1018  Replicas: 1018,1016,1017  Isr: 1018,1017,1016  Topic:
mno.de.pola.trump  Partition: 2  Leader: 1017  Replicas: 1016,1017,1018  Isr: 1016,1018.1017  Topic:
mno.de.pola.trump  Partition: 3  Leader: 1017  Replicas: 1017,1016,1018  Isr: 1017,1018,1016  Topic:
mno.de.pola.trump  Partition: 4  Leader: 1018  Replicas: 1018,1017,1016  Isr: 1018,1017,1016  Topic:
mno.de.pola.trump  Partition: 5  Leader: 1016  Replicas: 1016,1018,1017  Isr: 1016,1018,1017  Topic:
mno.de.pola.trump  Partition: 6  Leader: 1017  Replicas: 1017,1018,1016  Isr: 1017,1018,1016

在以下示例中,我们可以看到副本服务器中缺少经纪人

/usr/hdp/current/kafka-broker/bin/kafka-topics.sh  --describe --zookeeper  master01:2181 --topic mno.de.pola.trump  Topic:mno.de.pola.trump  PartitionCount:100  ReplicationFactor:3  Configs:  Topic:
mno.de.pola.trump  Partition: 0  Leader: 1017  Replicas: 1017,1018       Isr: 1017,1018,1016  Topic:
mno.de.pola.trump  Partition: 1  Leader: 1018  Replicas: 1018,1016,1017  Isr: 1018,1017,1016  Topic:
mno.de.pola.trump  Partition: 2  Leader: 1017  Replicas: 1016,1017,1018  Isr: 1016,1018,1017  Topic:
mno.de.pola.trump  Partition: 3  Leader: 1017  Replicas: 1017,1016       Isr: 1017,1018,1016  Topic:
mno.de.pola.trump  Partition: 4  Leader: 1018  Replicas: 1018,1017,1016  Isr: 1018,1017,1016  Topic:
mno.de.pola.trump  Partition: 5  Leader: 1016  Replicas: 1016,1018,1017  Isr: 1016,1018,1017  Topic:
mno.de.pola.trump  Partition: 6  Leader: 1017 Replicas:  1017,1018,1016  Isr: 1017,1018,1016

依此类推

所以我的问题是: 我很高兴知道一些可以验证kafka-topics.sh --describe输出的工具(脚本) 关于这个的任何想法

平衡配置良好的示例

/usr/hdp/current/kafka-broker/bin/kafka-topics.sh  --describe --zookeeper  master01:2181 --topic mno.de.pola.trump

        Topic: mno.de.pola.trump      PartitionCount:100      ReplicationFactor:3     Configs:
        Topic: mno.de.pola.trump     Partition: 0    Leader: 1017    Replicas: 1017,1018,1016        Isr: 1017,1018,1016
        Topic: mno.de.pola.trump     Partition: 1    Leader: 1018    Replicas: 1018,1016,1017        Isr: 1018,1017,1016
        Topic: mno.de.pola.trump     Partition: 2    Leader: 1016    Replicas: 1016,1017,1018        Isr: 1016,1018,1017
        Topic: mno.de.pola.trump     Partition: 3    Leader: 1017    Replicas: 1017,1016,1018        Isr: 1017,1018,1016
        Topic: mno.de.pola.trump     Partition: 4    Leader: 1018    Replicas: 1018,1017,1016        Isr: 1018,1017,1016
        Topic: mno.de.pola.trump     Partition: 5    Leader: 1016    Replicas: 1016,1018,1017        Isr: 1016,1018,1017

0 个答案:

没有答案