如何将带有-1的分区移动到另一个代理?

时间:2016-11-08 03:47:55

标签: apache-kafka

让我们说所有具有分区的领导者和副本的节点崩溃,节点不可恢复。

如何选择另一个Kafka经纪人作为该分区的领导者? (我对数据丢失很好)

我试过" kafka-preferred-replica-election.sh"和" kafka-reassign-partitions.sh"但没有成功。

设定:

  • 两个节点kafka群集,代理ID为10和11。
  • 主题"测试"复制因子1和分区1.经纪人11是主题的领导者" test"分区0
  • 经纪人11倒闭。
  • Kafka版本 - kafka_2.11-0.10.1.0

主题描述

Topic:test  PartitionCount:1    ReplicationFactor:1 Configs:
Topic: test Partition: 0    Leader: -1  Replicas: 11    Isr:

kafka-reassign-partitions.sh输出:

bin/kafka-reassign-partitions.sh --zookeeper myvm:2181 --reassignment-json-file manual-assignment.json --execute
Partitions reassignment failed due to The proposed assignment contains non-existent partitions: List([test,10])
kafka.common.AdminCommandFailedException: The proposed assignment contains non-existent partitions: List([test,10])
    at kafka.admin.ReassignPartitionsCommand$.parseAndValidate(ReassignPartitionsCommand.scala:201)
    at kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:153)
    at kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:149)
    at kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:46)
    at kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)

cat manual-assignment.json
{
 "partitions":
  [
    {"topic": "test", "partition": 10, "replicas": [10]}
  ],
  "version":1
}

kafka-preferred-replica-election.sh

bin/kafka-preferred-replica-election.sh --zookeeper myvm:2181 --path-to-json-file topicPartitionList.json
[2016-11-08 03:27:01,993] ERROR Skipping preferred replica leader election for partition [test,10] since it doesn't exist (kafka.admin.PreferredReplicaLeaderElectionCommand)
Successfully started preferred replica election for partitions Set([test,10])

cat topicPartitionList.json
{
 "partitions":
  [
    {"topic": "test", "partition": 10}
  ]
}

1 个答案:

答案 0 :(得分:1)

卡夫卡只从ISR中选出一名领导并分配复制品(当启用不洁领导人选举时)。对于复制因子设置为1的情况,在领导者代理崩溃后,ISR中没有代理。没有候选人可以当选为新的领导者,所以这就是设计。