我遇到了一个有线问题,当我检查主题的状态时,我发现__consumer_offsets有100个分区,而当我检查配置时,它只有50个分区,所以我不知道发生这种情况的原因,我的kafka版本是0.11.0.2。我曾经尝试查看是否有人扩展了__consumer_offsets,但实际上该主题无法扩展为: 执行主题命令时出错:不能更改偏移量主题的分区数。
[2018-12-27 01:16:25,422] ERROR java.lang.IllegalArgumentException: The number of partitions for the offsets topic cannot be changed.
at kafka.admin.TopicCommand$$anonfun$alterTopic$1.apply(TopicCommand.scala:141)
at kafka.admin.TopicCommand$$anonfun$alterTopic$1.apply(TopicCommand.scala:124)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at kafka.admin.TopicCommand$.alterTopic(TopicCommand.scala:124)
at kafka.admin.TopicCommand$.main(TopicCommand.scala:64)
at kafka.admin.TopicCommand.main(TopicCommand.scala)
(kafka.admin.TopicCommand$)
我的配置:
offset.metadata.max.bytes = 4096
offsets.commit.required.acks = -1
offsets.commit.timeout.ms = 5000
offsets.load.buffer.size = 5242880
offsets.retention.check.interval.ms = 600000
offsets.retention.minutes = 1440
offsets.topic.compression.codec = 0
offsets.topic.num.partitions = 50
offsets.topic.replication.factor = 3
offsets.topic.segment.bytes = 104857600
我的主题结果是:
[data@dt-kafka2-01 kafka]$ bin/kafka-topics.sh --describe --topic __consumer_offsets --zookeeper dt-kafka2-zk01:2181
Topic:__consumer_offsets PartitionCount:100 ReplicationFactor:3 Configs:segment.bytes=104857600,cleanup.policy=compact,compression.type=producer
Topic: __consumer_offsets Partition: 0 Leader: 12 Replicas: 12,4,5 Isr: 5,12,4
Topic: __consumer_offsets Partition: 1 Leader: 13 Replicas: 13,5,6 Isr: 5,6,13
Topic: __consumer_offsets Partition: 2 Leader: 14 Replicas: 14,6,7 Isr: 14,6,7
Topic: __consumer_offsets Partition: 3 Leader: 15 Replicas: 15,7,8 Isr: 7,8,15
Topic: __consumer_offsets Partition: 4 Leader: 16 Replicas: 16,8,9 Isr: 9,16,8
Topic: __consumer_offsets Partition: 5 Leader: 17 Replicas: 17,9,10 Isr: 10,9,17
Topic: __consumer_offsets Partition: 6 Leader: 18 Replicas: 18,10,11 Isr: 10,18,11
Topic: __consumer_offsets Partition: 7 Leader: 19 Replicas: 19,11,12 Isr: 12,11,19
Topic: __consumer_offsets Partition: 8 Leader: 20 Replicas: 20,12,13 Isr: 20,12,13
Topic: __consumer_offsets Partition: 9 Leader: 1 Replicas: 1,13,14 Isr: 14,1,13
Topic: __consumer_offsets Partition: 10 Leader: 2 Replicas: 2,14,15 Isr: 14,2,15
Topic: __consumer_offsets Partition: 11 Leader: 3 Replicas: 3,15,16 Isr: 3,16,15
Topic: __consumer_offsets Partition: 12 Leader: 4 Replicas: 4,16,17 Isr: 17,16,4
Topic: __consumer_offsets Partition: 13 Leader: 5 Replicas: 5,17,18 Isr: 5,17,18
Topic: __consumer_offsets Partition: 14 Leader: 6 Replicas: 6,18,19 Isr: 6,18,19
Topic: __consumer_offsets Partition: 15 Leader: 7 Replicas: 7,19,20 Isr: 20,7,19
Topic: __consumer_offsets Partition: 16 Leader: 8 Replicas: 8,20,1 Isr: 20,1,8
Topic: __consumer_offsets Partition: 17 Leader: 1 Replicas: 9,1,2 Isr: 1,9,2
Topic: __consumer_offsets Partition: 18 Leader: 2 Replicas: 10,2,3 Isr: 10,2,3
Topic: __consumer_offsets Partition: 19 Leader: 11 Replicas: 11,3,4 Isr: 3,4,11
Topic: __consumer_offsets Partition: 20 Leader: 12 Replicas: 12,5,6 Isr: 5,6,12
Topic: __consumer_offsets Partition: 21 Leader: 13 Replicas: 13,6,7 Isr: 6,7,13
Topic: __consumer_offsets Partition: 22 Leader: 14 Replicas: 14,7,8 Isr: 14,7,8
Topic: __consumer_offsets Partition: 23 Leader: 15 Replicas: 15,8,9 Isr: 9,8,15
Topic: __consumer_offsets Partition: 24 Leader: 16 Replicas: 16,9,10 Isr: 10,9,16
Topic: __consumer_offsets Partition: 25 Leader: 17 Replicas: 17,10,11 Isr: 10,17,11
Topic: __consumer_offsets Partition: 26 Leader: 18 Replicas: 18,11,12 Isr: 12,18,11
Topic: __consumer_offsets Partition: 27 Leader: 19 Replicas: 19,12,13 Isr: 12,19,13
Topic: __consumer_offsets Partition: 28 Leader: 20 Replicas: 20,13,14 Isr: 14,20,13
Topic: __consumer_offsets Partition: 29 Leader: 1 Replicas: 1,14,15 Isr: 14,1,15
Topic: __consumer_offsets Partition: 30 Leader: 2 Replicas: 2,15,16 Isr: 2,16,15
Topic: __consumer_offsets Partition: 31 Leader: 3 Replicas: 3,16,17 Isr: 17,3,16
Topic: __consumer_offsets Partition: 32 Leader: 4 Replicas: 4,17,18 Isr: 17,18,4
Topic: __consumer_offsets Partition: 33 Leader: 5 Replicas: 5,18,19 Isr: 5,18,19
Topic: __consumer_offsets Partition: 34 Leader: 6 Replicas: 6,19,20 Isr: 20,6,19
Topic: __consumer_offsets Partition: 35 Leader: 7 Replicas: 7,20,1 Isr: 20,1,7
Topic: __consumer_offsets Partition: 36 Leader: 8 Replicas: 8,1,2 Isr: 1,2,8
Topic: __consumer_offsets Partition: 37 Leader: 9 Replicas: 9,2,3 Isr: 2,3,9
Topic: __consumer_offsets Partition: 38 Leader: 10 Replicas: 10,3,4 Isr: 3,4,10
Topic: __consumer_offsets Partition: 39 Leader: 11 Replicas: 11,4,5 Isr: 5,4,11
Topic: __consumer_offsets Partition: 40 Leader: 12 Replicas: 12,6,7 Isr: 6,12,7
Topic: __consumer_offsets Partition: 41 Leader: 13 Replicas: 13,7,8 Isr: 7,8,13
Topic: __consumer_offsets Partition: 42 Leader: 14 Replicas: 14,8,9 Isr: 14,9,8
Topic: __consumer_offsets Partition: 43 Leader: 15 Replicas: 15,9,10 Isr: 10,9,15
Topic: __consumer_offsets Partition: 44 Leader: 16 Replicas: 16,10,11 Isr: 10,16,11
Topic: __consumer_offsets Partition: 45 Leader: 17 Replicas: 17,11,12 Isr: 17,12,11
Topic: __consumer_offsets Partition: 46 Leader: 18 Replicas: 18,12,13 Isr: 12,18,13
Topic: __consumer_offsets Partition: 47 Leader: 19 Replicas: 19,13,14 Isr: 14,19,13
Topic: __consumer_offsets Partition: 48 Leader: 20 Replicas: 20,14,15 Isr: 14,20,15
Topic: __consumer_offsets Partition: 49 Leader: 1 Replicas: 1,15,16 Isr: 1,16,15
Topic: __consumer_offsets Partition: 50 Leader: 2 Replicas: 2,16,17 Isr: 2,17,16
Topic: __consumer_offsets Partition: 51 Leader: 3 Replicas: 3,17,18 Isr: 17,3,18
Topic: __consumer_offsets Partition: 52 Leader: 4 Replicas: 4,18,19 Isr: 18,4,19
Topic: __consumer_offsets Partition: 53 Leader: 5 Replicas: 5,19,20 Isr: 5,20,19
Topic: __consumer_offsets Partition: 54 Leader: 6 Replicas: 6,20,1 Isr: 20,1,6
Topic: __consumer_offsets Partition: 55 Leader: 7 Replicas: 7,1,2 Isr: 1,2,7
Topic: __consumer_offsets Partition: 56 Leader: 8 Replicas: 8,2,3 Isr: 2,3,8
Topic: __consumer_offsets Partition: 57 Leader: 9 Replicas: 9,3,4 Isr: 9,3,4
Topic: __consumer_offsets Partition: 58 Leader: 10 Replicas: 10,4,5 Isr: 5,10,4
Topic: __consumer_offsets Partition: 59 Leader: 11 Replicas: 11,5,6 Isr: 5,6,11
Topic: __consumer_offsets Partition: 60 Leader: 12 Replicas: 12,7,8 Isr: 12,7,8
Topic: __consumer_offsets Partition: 61 Leader: 13 Replicas: 13,8,9 Isr: 9,8,13
Topic: __consumer_offsets Partition: 62 Leader: 14 Replicas: 14,9,10 Isr: 10,14,9
Topic: __consumer_offsets Partition: 63 Leader: 15 Replicas: 15,10,11 Isr: 10,15,11
Topic: __consumer_offsets Partition: 64 Leader: 16 Replicas: 16,11,12 Isr: 12,16,11
Topic: __consumer_offsets Partition: 65 Leader: 17 Replicas: 17,12,13 Isr: 17,12,13
Topic: __consumer_offsets Partition: 66 Leader: 18 Replicas: 18,13,14 Isr: 14,18,13
Topic: __consumer_offsets Partition: 67 Leader: 19 Replicas: 19,14,15 Isr: 14,15,19
Topic: __consumer_offsets Partition: 68 Leader: 20 Replicas: 20,15,16 Isr: 20,16,15
Topic: __consumer_offsets Partition: 69 Leader: 1 Replicas: 1,16,17 Isr: 1,17,16
Topic: __consumer_offsets Partition: 70 Leader: 2 Replicas: 2,17,18 Isr: 2,17,18
Topic: __consumer_offsets Partition: 71 Leader: 3 Replicas: 3,18,19 Isr: 3,18,19
Topic: __consumer_offsets Partition: 72 Leader: 4 Replicas: 4,19,20 Isr: 20,4,19
Topic: __consumer_offsets Partition: 73 Leader: 5 Replicas: 5,20,1 Isr: 5,20,1
Topic: __consumer_offsets Partition: 74 Leader: 6 Replicas: 6,1,2 Isr: 1,6,2
Topic: __consumer_offsets Partition: 75 Leader: 7 Replicas: 7,2,3 Isr: 2,7,3
Topic: __consumer_offsets Partition: 76 Leader: 8 Replicas: 8,3,4 Isr: 3,8,4
Topic: __consumer_offsets Partition: 77 Leader: 9 Replicas: 9,4,5 Isr: 5,9,4
Topic: __consumer_offsets Partition: 78 Leader: 10 Replicas: 10,5,6 Isr: 5,10,6
Topic: __consumer_offsets Partition: 79 Leader: 11 Replicas: 11,6,7 Isr: 6,7,11
Topic: __consumer_offsets Partition: 80 Leader: 12 Replicas: 12,8,9 Isr: 9,12,8
Topic: __consumer_offsets Partition: 81 Leader: 13 Replicas: 13,9,10 Isr: 10,9,13
Topic: __consumer_offsets Partition: 82 Leader: 14 Replicas: 14,10,11 Isr: 10,14,11
Topic: __consumer_offsets Partition: 83 Leader: 15 Replicas: 15,11,12 Isr: 12,15,11
Topic: __consumer_offsets Partition: 84 Leader: 16 Replicas: 16,12,13 Isr: 12,16,13
Topic: __consumer_offsets Partition: 85 Leader: 17 Replicas: 17,13,14 Isr: 14,17,13
Topic: __consumer_offsets Partition: 86 Leader: 18 Replicas: 18,14,15 Isr: 14,18,15
Topic: __consumer_offsets Partition: 87 Leader: 19 Replicas: 19,15,16 Isr: 16,15,19
Topic: __consumer_offsets Partition: 88 Leader: 20 Replicas: 20,16,17 Isr: 20,17,16
Topic: __consumer_offsets Partition: 89 Leader: 1 Replicas: 1,17,18 Isr: 1,17,18
Topic: __consumer_offsets Partition: 90 Leader: 2 Replicas: 2,18,19 Isr: 2,18,19
Topic: __consumer_offsets Partition: 91 Leader: 3 Replicas: 3,19,20 Isr: 20,3,19
Topic: __consumer_offsets Partition: 92 Leader: 4 Replicas: 4,20,1 Isr: 20,1,4
Topic: __consumer_offsets Partition: 93 Leader: 5 Replicas: 5,1,2 Isr: 5,1,2
Topic: __consumer_offsets Partition: 94 Leader: 6 Replicas: 6,2,3 Isr: 6,2,3
Topic: __consumer_offsets Partition: 95 Leader: 7 Replicas: 7,3,4 Isr: 7,3,4
Topic: __consumer_offsets Partition: 96 Leader: 8 Replicas: 8,4,5 Isr: 5,8,4
Topic: __consumer_offsets Partition: 97 Leader: 9 Replicas: 9,5,6 Isr: 5,6,9
Topic: __consumer_offsets Partition: 98 Leader: 10 Replicas: 10,6,7 Isr: 10,6,7
Topic: __consumer_offsets Partition: 99 Leader: 11 Replicas: 11,7,8 Isr: 7,8,11
那么任何人都可以给出一些解释吗?