我在使用Kafka Manager时遇到Kafka分区分配问题
这是我的步骤
1)我有2个卡夫卡经纪人(1,2)
2)我创建经纪人3
3)我生成分区分配以将数据分区到broker2,broker3
一切都很好,直到broker1遇到意外错误
kk1_1 | [2019-03-12 09:36:42,738] ERROR Error while deleting log for __consumer_offsets-15 in dir /kafka/data (kafka.server.LogDirFailureChannel)
kk1_1 | java.nio.file.FileSystemException: /kafka/data/__consumer_offsets-15.374d3b51e2d5427b97c9b7dca63cde64-delete/.nfsa28bc636ec76204a00000727: Device or resource busy
此错误导致broker1损坏。
我能够重新启动broker1(在进行恢复过程后,例如
{"log":"[2019-03-13 03:35:16,580] WARN Found a corrupted index file due to requirement failed: Corrupt index found, index file (/kafka/data/json.candidate-portal.candidate-update-4/00000000000000001231.index) has non-zero size but the last offset is 1231 which is no larger than the base offset 1231.}. deleting /kafka/data/json.candidate-portal.candidate-update-4/00000000000000001231.timeindex, /kafka/data/json.candidate-portal.candidate-update-4/00000000000000001231.index, and /kafka/data/json.candidate-portal.candidate-update-4/00000000000000001231.txnindex and rebuilding index... (kafka.log.Log)\n","stream":"stdout","time":"2019-03-13T03:35:16.58110204Z"}
)
我找到的问题
重新启动Broker1之后,broker2和broker3要求__consumer_offsets-15
被broker1删除
分区分配尚未完成,我的集群无法接收消息。
我通过将/ kafka / data / __ consumer_offsets-15从broker2复制到broker1并重新启动broker3作弊
然后它起作用了(!!!)。除__consumer_offsets-15
从卡夫卡经理的角度来看, partition assignments incomplete
failed replica 我的问题
1)问题是什么?
2)是否有使分区分配恢复正常的方法?这样我就可以创建另一个分区分配。 (例如,添加新的经纪人)
3)我可以删除__consumer_offset_15
感谢您的帮助