卡夫卡州不一致

时间:2019-03-15 11:05:54

标签: apache-kafka

我在使用Kafka Manager时遇到Kafka分区分配问题

这是我的步骤

1)我有2个卡夫卡经纪人(1,2)

2)我创建经纪人3

3)我生成分区分配以将数据分区到broker2,broker3

一切都很好,直到broker1遇到意外错误

kk1_1  | [2019-03-12 09:36:42,738] ERROR Error while deleting log for __consumer_offsets-15 in dir /kafka/data (kafka.server.LogDirFailureChannel)
kk1_1  | java.nio.file.FileSystemException: /kafka/data/__consumer_offsets-15.374d3b51e2d5427b97c9b7dca63cde64-delete/.nfsa28bc636ec76204a00000727: Device or resource busy

此错误导致broker1损坏。

我能够重新启动broker1(在进行恢复过程后,例如

{"log":"[2019-03-13 03:35:16,580] WARN Found a corrupted index file due to requirement failed: Corrupt index found, index file (/kafka/data/json.candidate-portal.candidate-update-4/00000000000000001231.index) has non-zero size but the last offset is 1231 which is no larger than the base offset 1231.}. deleting /kafka/data/json.candidate-portal.candidate-update-4/00000000000000001231.timeindex, /kafka/data/json.candidate-portal.candidate-update-4/00000000000000001231.index, and /kafka/data/json.candidate-portal.candidate-update-4/00000000000000001231.txnindex and rebuilding index... (kafka.log.Log)\n","stream":"stdout","time":"2019-03-13T03:35:16.58110204Z"}
) 

我找到的问题

重新启动Broker1之后,broker2和broker3要求__consumer_offsets-15被broker1删除

分区分配尚未完成,我的集群无法接收消息。

我通过将/ kafka / data / __ consumer_offsets-15从broker2复制到broker1并重新启动broker3作弊

然后它起作用了(!!!)。除__consumer_offsets-15

以外的大多数主题都已完成分区分配

从卡夫卡经理的角度来看, partition assignments incomplete

failed replica 我的问题

1)问题是什么?

2)是否有使分区分配恢复正常的方法?这样我就可以创建另一个分区分配。 (例如,添加新的经纪人)

3)我可以删除__consumer_offset_15

感谢您的帮助

0 个答案:

没有答案