Question

我的目标是在消费者组中寻找所有消费者，直到每个分区的开始。我使用spring-kafka并尝试以以下方式使用ConsumerSeekAware界面：

override fun onPartitionsAssigned(assignments: MutableMap<TopicPartition, Long>?, callback: ConsumerSeekAware.ConsumerSeekCallback?) {
  logger.info { "on partitions assigned" }
  assignments?.forEach { topic, _ -> callback?.seekToBeginning(topic.topic(), topic.partition()) }
}

现在，对于一个组中只有一个消费者，它起作用了，但是，如果我想寻求一个以上的消费者，该怎么办？我尝试了一下，最终导致分区的无限重新平衡。我会误解什么？

Answer 1

我最终使用了ConsumerSeekAware的其他方法，即registerSeekCallback()。调用onPartitionsAssigned之后，我将当前分区另存为侦听器中的成员。调用registerSeekCallback之后，我将回调保存在实例中。

然后，我可以针对每个使用者（通过REST或启动后）按需调用consumeFromBeginning：

class EventConsumer : ConsumerSeekAware {

    private lateinit var assignedTopicPartitions: List<TopicPartition>
    private lateinit var consumerSeekCallback: ConsumerSeekAware.ConsumerSeekCallback

    fun consumeFromBeginning() {
        assignedTopicPartitions.forEach { topicPartition ->
            consumerSeekCallback.seekToBeginning(topicPartition.topic(), topicPartition.partition())}
    }

    override fun onPartitionsAssigned(assignments: MutableMap<TopicPartition, Long>?, callback: ConsumerSeekAware.ConsumerSeekCallback?) {
        logger.debug { "on partitions assigned" }
        this.assignedTopicPartitions = assignments?.map { it.key } ?: error("no assignments found")
    }

    override fun registerSeekCallback(callback: ConsumerSeekAware.ConsumerSeekCallback?) {
        logger.debug { "registerSeekCallback" }
        this.consumerSeekCallback = callback ?: error("seek callback not found")
    }

    override fun onIdleContainer(assignments: MutableMap<TopicPartition, Long>?, callback: ConsumerSeekAware.ConsumerSeekCallback?) {}
}

适当地寻求组中的所有消费者

1 个答案: