卡夫卡__consumer_offsets分区num与配置值不匹配

时间:2018-12-27 09:40:50

标签: apache-kafka

我遇到了一个有线问题,当我检查主题的状态时,我发现__consumer_offsets有100个分区,而当我检查配置时,它只有50个分区,所以我不知道发生这种情况的原因,我的kafka版本是0.11.0.2。我曾经尝试查看是否有人扩展了__consumer_offsets,但实际上该主题无法扩展为: 执行主题命令时出错:不能更改偏移量主题的分区数。

[2018-12-27 01:16:25,422] ERROR java.lang.IllegalArgumentException: The number of partitions for the offsets topic cannot be changed.
    at kafka.admin.TopicCommand$$anonfun$alterTopic$1.apply(TopicCommand.scala:141)
    at kafka.admin.TopicCommand$$anonfun$alterTopic$1.apply(TopicCommand.scala:124)
    at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
    at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
    at kafka.admin.TopicCommand$.alterTopic(TopicCommand.scala:124)
    at kafka.admin.TopicCommand$.main(TopicCommand.scala:64)
    at kafka.admin.TopicCommand.main(TopicCommand.scala)
 (kafka.admin.TopicCommand$)

我的配置:

        offset.metadata.max.bytes = 4096
        offsets.commit.required.acks = -1
        offsets.commit.timeout.ms = 5000
        offsets.load.buffer.size = 5242880
        offsets.retention.check.interval.ms = 600000
        offsets.retention.minutes = 1440
        offsets.topic.compression.codec = 0
        offsets.topic.num.partitions = 50
        offsets.topic.replication.factor = 3
        offsets.topic.segment.bytes = 104857600

我的主题结果是:

[data@dt-kafka2-01 kafka]$ bin/kafka-topics.sh --describe --topic __consumer_offsets --zookeeper dt-kafka2-zk01:2181
Topic:__consumer_offsets    PartitionCount:100  ReplicationFactor:3 Configs:segment.bytes=104857600,cleanup.policy=compact,compression.type=producer
    Topic: __consumer_offsets   Partition: 0    Leader: 12  Replicas: 12,4,5    Isr: 5,12,4
    Topic: __consumer_offsets   Partition: 1    Leader: 13  Replicas: 13,5,6    Isr: 5,6,13
    Topic: __consumer_offsets   Partition: 2    Leader: 14  Replicas: 14,6,7    Isr: 14,6,7
    Topic: __consumer_offsets   Partition: 3    Leader: 15  Replicas: 15,7,8    Isr: 7,8,15
    Topic: __consumer_offsets   Partition: 4    Leader: 16  Replicas: 16,8,9    Isr: 9,16,8
    Topic: __consumer_offsets   Partition: 5    Leader: 17  Replicas: 17,9,10   Isr: 10,9,17
    Topic: __consumer_offsets   Partition: 6    Leader: 18  Replicas: 18,10,11  Isr: 10,18,11
    Topic: __consumer_offsets   Partition: 7    Leader: 19  Replicas: 19,11,12  Isr: 12,11,19
    Topic: __consumer_offsets   Partition: 8    Leader: 20  Replicas: 20,12,13  Isr: 20,12,13
    Topic: __consumer_offsets   Partition: 9    Leader: 1   Replicas: 1,13,14   Isr: 14,1,13
    Topic: __consumer_offsets   Partition: 10   Leader: 2   Replicas: 2,14,15   Isr: 14,2,15
    Topic: __consumer_offsets   Partition: 11   Leader: 3   Replicas: 3,15,16   Isr: 3,16,15
    Topic: __consumer_offsets   Partition: 12   Leader: 4   Replicas: 4,16,17   Isr: 17,16,4
    Topic: __consumer_offsets   Partition: 13   Leader: 5   Replicas: 5,17,18   Isr: 5,17,18
    Topic: __consumer_offsets   Partition: 14   Leader: 6   Replicas: 6,18,19   Isr: 6,18,19
    Topic: __consumer_offsets   Partition: 15   Leader: 7   Replicas: 7,19,20   Isr: 20,7,19
    Topic: __consumer_offsets   Partition: 16   Leader: 8   Replicas: 8,20,1    Isr: 20,1,8
    Topic: __consumer_offsets   Partition: 17   Leader: 1   Replicas: 9,1,2 Isr: 1,9,2
    Topic: __consumer_offsets   Partition: 18   Leader: 2   Replicas: 10,2,3    Isr: 10,2,3
    Topic: __consumer_offsets   Partition: 19   Leader: 11  Replicas: 11,3,4    Isr: 3,4,11
    Topic: __consumer_offsets   Partition: 20   Leader: 12  Replicas: 12,5,6    Isr: 5,6,12
    Topic: __consumer_offsets   Partition: 21   Leader: 13  Replicas: 13,6,7    Isr: 6,7,13
    Topic: __consumer_offsets   Partition: 22   Leader: 14  Replicas: 14,7,8    Isr: 14,7,8
    Topic: __consumer_offsets   Partition: 23   Leader: 15  Replicas: 15,8,9    Isr: 9,8,15
    Topic: __consumer_offsets   Partition: 24   Leader: 16  Replicas: 16,9,10   Isr: 10,9,16
    Topic: __consumer_offsets   Partition: 25   Leader: 17  Replicas: 17,10,11  Isr: 10,17,11
    Topic: __consumer_offsets   Partition: 26   Leader: 18  Replicas: 18,11,12  Isr: 12,18,11
    Topic: __consumer_offsets   Partition: 27   Leader: 19  Replicas: 19,12,13  Isr: 12,19,13
    Topic: __consumer_offsets   Partition: 28   Leader: 20  Replicas: 20,13,14  Isr: 14,20,13
    Topic: __consumer_offsets   Partition: 29   Leader: 1   Replicas: 1,14,15   Isr: 14,1,15
    Topic: __consumer_offsets   Partition: 30   Leader: 2   Replicas: 2,15,16   Isr: 2,16,15
    Topic: __consumer_offsets   Partition: 31   Leader: 3   Replicas: 3,16,17   Isr: 17,3,16
    Topic: __consumer_offsets   Partition: 32   Leader: 4   Replicas: 4,17,18   Isr: 17,18,4
    Topic: __consumer_offsets   Partition: 33   Leader: 5   Replicas: 5,18,19   Isr: 5,18,19
    Topic: __consumer_offsets   Partition: 34   Leader: 6   Replicas: 6,19,20   Isr: 20,6,19
    Topic: __consumer_offsets   Partition: 35   Leader: 7   Replicas: 7,20,1    Isr: 20,1,7
    Topic: __consumer_offsets   Partition: 36   Leader: 8   Replicas: 8,1,2 Isr: 1,2,8
    Topic: __consumer_offsets   Partition: 37   Leader: 9   Replicas: 9,2,3 Isr: 2,3,9
    Topic: __consumer_offsets   Partition: 38   Leader: 10  Replicas: 10,3,4    Isr: 3,4,10
    Topic: __consumer_offsets   Partition: 39   Leader: 11  Replicas: 11,4,5    Isr: 5,4,11
    Topic: __consumer_offsets   Partition: 40   Leader: 12  Replicas: 12,6,7    Isr: 6,12,7
    Topic: __consumer_offsets   Partition: 41   Leader: 13  Replicas: 13,7,8    Isr: 7,8,13
    Topic: __consumer_offsets   Partition: 42   Leader: 14  Replicas: 14,8,9    Isr: 14,9,8
    Topic: __consumer_offsets   Partition: 43   Leader: 15  Replicas: 15,9,10   Isr: 10,9,15
    Topic: __consumer_offsets   Partition: 44   Leader: 16  Replicas: 16,10,11  Isr: 10,16,11
    Topic: __consumer_offsets   Partition: 45   Leader: 17  Replicas: 17,11,12  Isr: 17,12,11
    Topic: __consumer_offsets   Partition: 46   Leader: 18  Replicas: 18,12,13  Isr: 12,18,13
    Topic: __consumer_offsets   Partition: 47   Leader: 19  Replicas: 19,13,14  Isr: 14,19,13
    Topic: __consumer_offsets   Partition: 48   Leader: 20  Replicas: 20,14,15  Isr: 14,20,15
    Topic: __consumer_offsets   Partition: 49   Leader: 1   Replicas: 1,15,16   Isr: 1,16,15
    Topic: __consumer_offsets   Partition: 50   Leader: 2   Replicas: 2,16,17   Isr: 2,17,16
    Topic: __consumer_offsets   Partition: 51   Leader: 3   Replicas: 3,17,18   Isr: 17,3,18
    Topic: __consumer_offsets   Partition: 52   Leader: 4   Replicas: 4,18,19   Isr: 18,4,19
    Topic: __consumer_offsets   Partition: 53   Leader: 5   Replicas: 5,19,20   Isr: 5,20,19
    Topic: __consumer_offsets   Partition: 54   Leader: 6   Replicas: 6,20,1    Isr: 20,1,6
    Topic: __consumer_offsets   Partition: 55   Leader: 7   Replicas: 7,1,2 Isr: 1,2,7
    Topic: __consumer_offsets   Partition: 56   Leader: 8   Replicas: 8,2,3 Isr: 2,3,8
    Topic: __consumer_offsets   Partition: 57   Leader: 9   Replicas: 9,3,4 Isr: 9,3,4
    Topic: __consumer_offsets   Partition: 58   Leader: 10  Replicas: 10,4,5    Isr: 5,10,4
    Topic: __consumer_offsets   Partition: 59   Leader: 11  Replicas: 11,5,6    Isr: 5,6,11
    Topic: __consumer_offsets   Partition: 60   Leader: 12  Replicas: 12,7,8    Isr: 12,7,8
    Topic: __consumer_offsets   Partition: 61   Leader: 13  Replicas: 13,8,9    Isr: 9,8,13
    Topic: __consumer_offsets   Partition: 62   Leader: 14  Replicas: 14,9,10   Isr: 10,14,9
    Topic: __consumer_offsets   Partition: 63   Leader: 15  Replicas: 15,10,11  Isr: 10,15,11
    Topic: __consumer_offsets   Partition: 64   Leader: 16  Replicas: 16,11,12  Isr: 12,16,11
    Topic: __consumer_offsets   Partition: 65   Leader: 17  Replicas: 17,12,13  Isr: 17,12,13
    Topic: __consumer_offsets   Partition: 66   Leader: 18  Replicas: 18,13,14  Isr: 14,18,13
    Topic: __consumer_offsets   Partition: 67   Leader: 19  Replicas: 19,14,15  Isr: 14,15,19
    Topic: __consumer_offsets   Partition: 68   Leader: 20  Replicas: 20,15,16  Isr: 20,16,15
    Topic: __consumer_offsets   Partition: 69   Leader: 1   Replicas: 1,16,17   Isr: 1,17,16
    Topic: __consumer_offsets   Partition: 70   Leader: 2   Replicas: 2,17,18   Isr: 2,17,18
    Topic: __consumer_offsets   Partition: 71   Leader: 3   Replicas: 3,18,19   Isr: 3,18,19
    Topic: __consumer_offsets   Partition: 72   Leader: 4   Replicas: 4,19,20   Isr: 20,4,19
    Topic: __consumer_offsets   Partition: 73   Leader: 5   Replicas: 5,20,1    Isr: 5,20,1
    Topic: __consumer_offsets   Partition: 74   Leader: 6   Replicas: 6,1,2 Isr: 1,6,2
    Topic: __consumer_offsets   Partition: 75   Leader: 7   Replicas: 7,2,3 Isr: 2,7,3
    Topic: __consumer_offsets   Partition: 76   Leader: 8   Replicas: 8,3,4 Isr: 3,8,4
    Topic: __consumer_offsets   Partition: 77   Leader: 9   Replicas: 9,4,5 Isr: 5,9,4
    Topic: __consumer_offsets   Partition: 78   Leader: 10  Replicas: 10,5,6    Isr: 5,10,6
    Topic: __consumer_offsets   Partition: 79   Leader: 11  Replicas: 11,6,7    Isr: 6,7,11
    Topic: __consumer_offsets   Partition: 80   Leader: 12  Replicas: 12,8,9    Isr: 9,12,8
    Topic: __consumer_offsets   Partition: 81   Leader: 13  Replicas: 13,9,10   Isr: 10,9,13
    Topic: __consumer_offsets   Partition: 82   Leader: 14  Replicas: 14,10,11  Isr: 10,14,11
    Topic: __consumer_offsets   Partition: 83   Leader: 15  Replicas: 15,11,12  Isr: 12,15,11
    Topic: __consumer_offsets   Partition: 84   Leader: 16  Replicas: 16,12,13  Isr: 12,16,13
    Topic: __consumer_offsets   Partition: 85   Leader: 17  Replicas: 17,13,14  Isr: 14,17,13
    Topic: __consumer_offsets   Partition: 86   Leader: 18  Replicas: 18,14,15  Isr: 14,18,15
    Topic: __consumer_offsets   Partition: 87   Leader: 19  Replicas: 19,15,16  Isr: 16,15,19
    Topic: __consumer_offsets   Partition: 88   Leader: 20  Replicas: 20,16,17  Isr: 20,17,16
    Topic: __consumer_offsets   Partition: 89   Leader: 1   Replicas: 1,17,18   Isr: 1,17,18
    Topic: __consumer_offsets   Partition: 90   Leader: 2   Replicas: 2,18,19   Isr: 2,18,19
    Topic: __consumer_offsets   Partition: 91   Leader: 3   Replicas: 3,19,20   Isr: 20,3,19
    Topic: __consumer_offsets   Partition: 92   Leader: 4   Replicas: 4,20,1    Isr: 20,1,4
    Topic: __consumer_offsets   Partition: 93   Leader: 5   Replicas: 5,1,2 Isr: 5,1,2
    Topic: __consumer_offsets   Partition: 94   Leader: 6   Replicas: 6,2,3 Isr: 6,2,3
    Topic: __consumer_offsets   Partition: 95   Leader: 7   Replicas: 7,3,4 Isr: 7,3,4
    Topic: __consumer_offsets   Partition: 96   Leader: 8   Replicas: 8,4,5 Isr: 5,8,4
    Topic: __consumer_offsets   Partition: 97   Leader: 9   Replicas: 9,5,6 Isr: 5,6,9
    Topic: __consumer_offsets   Partition: 98   Leader: 10  Replicas: 10,6,7    Isr: 10,6,7
    Topic: __consumer_offsets   Partition: 99   Leader: 11  Replicas: 11,7,8    Isr: 7,8,11

那么任何人都可以给出一些解释吗?

0 个答案:

没有答案