Question

使用Kafka Java API，我可以使用一个重新平衡监听器来寻找这个主题的开头（代码是Scala，Kafka API是Java）：

class SeekToBeginningRebalanceListener[K, V](val consumer: KafkaConsumer[K, V]) extends ConsumerRebalanceListener {
  override def onPartitionsAssigned(partitions: util.Collection[TopicPartition]): Unit = {
    for (tp <- partitions.asScala) {
      consumer.seekToBeginning(util.Arrays.asList(tp))
    }
  }

  override def onPartitionsRevoked(partitions: util.Collection[TopicPartition]): Unit = { }
}

我会在订阅这样的主题时使用它：

kafkaConsumer.subscribe(java.util.Arrays.asList(topicName), new SeekToBeginningRebalanceListener(kafkaConsumer))

我怎么能用confluent-kafka python api做到这一点？

我可以写一个类似的分区赋值回调函数，它被调用：

def on_assign_callback(consumer, topic_partitions):
    for topic_partition in topic_partitions:
        print("assigned to partition. topic={}. partition={}. offset={}. error={}".format(
            topic_partition.topic, topic_partition.partition, topic_partition.offset, topic_partition.error))

        # This runs but has no effect.
        topic_partition.offset = 0

但我找不到任何API来进行搜索。我该怎么做？

Answer 1

来自https://github.com/confluentinc/confluent-kafka-python/issues/11

这对我有用：

def on_assign (c, ps):
    for p in ps:
        p.offset=-2
    c.assign(ps)

c.subscribe(['test'], on_assign=on_assign)

目前尚未实现Confluent Kafka Python API的

seekToBeginning函数。

Kafka寻求从Python API开始？

1 个答案: