Question

我使用spring-kafka 1.1.3.RELEASE和kafka-clients 0.10.0.0，并且我想像这样在工厂中使用setConsumerRebalanceListener，但是我不知道如何让使用者使用来保存使用者分区。谢谢您的任何建议！

@Bean   

KafkaListenerContainerFactory<ConcurrentMessageListenerContainer<Integer, String>>
        kafkaListenerBatchContainerFactory() {
            ConcurrentKafkaListenerContainerFactory<Integer, String> factory =
                    new ConcurrentKafkaListenerContainerFactory<>();
            factory.setConsumerFactory(consumerFactory());
            factory.setConcurrency(3);
            factory.getContainerProperties().setPollTimeout(3000);
            factory.getContainerProperties().setConsumerTaskExecutor(execD());
            factory.getContainerProperties().setAckMode(AbstractMessageListenerContainer.AckMode.MANUAL_IMMEDIATE);

            factory.getContainerProperties().setSyncCommits(true);

            factory.getContainerProperties().setConsumerRebalanceListener(new ConsumerRebalanceListener() {

                @Override
                public void onPartitionsRevoked(Collection<TopicPartition> collection) {
                    for (TopicPartition partition:collection){
                    //TODO how to get consumer?    saveOffsetInExternalStore(consumer,partition.partition());
                    }
                }

                @Override
                public void onPartitionsAssigned(Collection<TopicPartition> collection) {
                     for (TopicPartition partition:collection){
                    //TODO how to get consumer?      
                       consumer.seek();
                    }
                }
            });
            factory.setBatchListener(true);

            return factory;
        }

我这样使用工厂：

 @KafkaListener(group = "CID_alikafka_B024",topicPattern = "data_.*",containerFactory = "kafkaListenerBatchContainerFactory")
    public void receive2(List<String> data,Acknowledgment acknowledgment,
                         @Header(KafkaHeaders.RECEIVED_PARTITION_ID) List<Integer> partitions,
                         @Header(KafkaHeaders.OFFSET) List<Long> offsets,
                         @Header(KafkaHeaders.RECEIVED_TOPIC) List<String> topicName,
                         @Header(KafkaHeaders.RECEIVED_MESSAGE_KEY) List<String> messageKeys) {
        logger.info("start of batch receive");

    }

我知道Spring Kafka 2.1.9具有这样的ConsumerAwareRebalanceListener，但我想使用Spring Kafka 1.1.3RELEASE兼容kafka 0.10.0.0，我们的kafka版本是0.10.0.0

containerProperties.setConsumerRebalanceListener(new ConsumerAwareRebalanceListener() {

    @Override
    public void onPartitionsRevokedBeforeCommit(Consumer<?, ?> consumer, Collection<TopicPartition> partitions) {
        // acknowledge any pending Acknowledgments (if using manual acks)
    }

    @Override
    public void onPartitionsRevokedAfterCommit(Consumer<?, ?> consumer, Collection<TopicPartition> partitions) {
        // ...
            store(consumer.position(partition));
        // ...
    }

    @Override
    public void onPartitionsAssigned(Collection<TopicPartition> partitions) {
        // ...
            consumer.seek(partition, offsetTracker.getOffset() + 1);
        // ...
    }
});

我的pom是：

 <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
            <version>1.1.3.RELEASE</version>
            <exclusions>
                <!-- exclude kafka version problem-->
                <exclusion>
                    <groupId>org.apache.kafka</groupId>
                    <artifactId>kafka-clients</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.kafka/kafka-clients -->
        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka-clients</artifactId>
            <version>0.10.0.0</version>
        </dependency>

Answer 1

1.1.3在Apache Kafka瞬息万变的世界中极为古老。您不能在那里访问消费者。

Spring for Apache Kafka在2.0中添加了<link href="https://cdn.jsdelivr.net/npm/daterangepicker/daterangepicker.css" rel="stylesheet"/> <script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script> <script src="https://cdn.jsdelivr.net/momentjs/latest/moment.min.js"></script> <script src="https://cdn.jsdelivr.net/npm/daterangepicker/daterangepicker.min.js"></script> <input type="text" class="date-picker"> <input type="text" class="date-picker-2">。当前版本是2.1.8。

如果您无法升级代理，则较新版本的Kafka客户端可以与较旧的代理进行通讯（但您应该这样做，0.10.0.0也很旧）。

请参见the documenation。

另请参阅project page，其中介绍了兼容性。在融合页面上：

经纪人0.10.0 基本客户端兼容性： Java：客户端<= 0.10.0或> = 0.10.2

setConsumerRebalanceListener如何获得消费者

1 个答案: