Spring-Kafka:使用使用者暂停/恢复功能时发生的重新平衡操作不符合文档规定

时间:2019-02-19 07:08:12

标签: apache-kafka spring-kafka

Spring-Kafka:在按照文档使用暂停/恢复方法暂停/恢复使用者的同时,当使用自动分配但它不起作用时,不应发生重新平衡,而是发生重新平衡。如何在不重新平衡一段时间后暂停/恢复消费者并保持轮询?

用例:消费者应暂停一段时间,并继续轮询以使心跳并在时间到后恢复,但在消费者暂停期间,Kafka不应重新平衡。

            System.out.println("Consumer[" + Thread.currentThread().getName() + "] Partition [" + topicPartition + "] stopped consumption.");
            consumer.pause(Collections.singleton(topicPartition));                    
            try {
                    Thread.sleep(60000);
                    consumer.resume(Collections.singleton(topicPartition));
                    System.out.println("Consumer[" + Thread.currentThread().getName() + "] Partition [" + topicPartition + "] resumed consumption.");
            } catch (InterruptedException e) {              
                       e.printStackTrace();
            }

日志: 2019-02-19 15:19:49.173 INFO 82272 --- [rTaskExecutor-1] o.a.k.c.c.internals.AbstractCoordinator:[Consumer clientId = consumer-2,groupId = customer](重新)加入组 2019-02-19 15:19:49.175 INFO 82272 --- [rTaskExecutor-2] o.a.k.c.c.internals.AbstractCoordinator:[Consumer clientId = consumer-3,groupId = customer](重新)加入组 2019-02-19 15:19:49.181信息82272 --- [rTaskExecutor-3] o.a.k.c.c.internals.AbstractCoordinator:[Consumer clientId = consumer-4,groupId = customer](重新)加入组

2019-02-19 15:19:49.192信息82272 --- [rTaskExecutor-1] o.a.k.c.c.internals.AbstractCoordinator:[Consumer clientId = consumer-2,groupId = customer]成功加入第581代的组 2019-02-19 15:19:49.192 INFO 82272 --- [rTaskExecutor-2] o.a.k.c.c.internals.AbstractCoordinator:[Consumer clientId = consumer-3,groupId = customer]成功加入第581代的组

2019-02-19 15:19:49.194信息82272 --- [rTaskExecutor-1] oakccinternals.ConsumerCoordinator:[Consumer clientId = consumer-2,groupId = customer]设置新分配的分区[spring-kafka-topic -2,spring-kafka-topic-0,spring-kafka-topic-1] 2019-02-19 15:19:49.194信息82272 --- [rTaskExecutor-2] oakccinternals.ConsumerCoordinator:[Consumer clientId = consumer-3,groupId = customer]设置新分配的分区[spring-kafka-topic-4, spring-kafka-topic-5,spring-kafka-topic-3] 2019-02-19 15:19:49.218信息82272 --- [rTaskExecutor-2] osklKafkaMessageListenerContainer:分配的分区:[spring-kafka-topic-4,spring-kafka-topic-5,spring-kafka-topic-3 ] 2019-02-19 15:19:49.219信息82272 --- [rTaskExecutor-1] osklKafkaMessageListenerContainer:分配的分区:[spring-kafka-topic-2,spring-kafka-topic-0,spring-kafka-topic-1 ] 2019-02-19 15:19:49.223信息82272 --- [main] o.s.b.w.embedded.tomcat.TomcatWebServer:Tomcat在端口8080(http)上启动,具有上下文路径'' 2019-02-19 15:19:49.233信息82272 --- [main] c.g.s.S.SpringKafkaSupportApplication:在3.43秒内启动SpringKafkaSupportApplication(JVM运行3.85秒) 消费者[customerTaskExecutor-1]收到消息[顾客(name =,phoneNumber = 20)] 消费者[customerTaskExecutor-2]收到消息[顾客(名称=测试6,电话号码= 6)] Consumer [customerTaskExecutor-1]分区[spring-kafka-topic-2]停止了消费。 Consumer [customerTaskExecutor-1]分区[spring-kafka-topic-1]停止了消费。 2019-02-19 15:19:52.200 INFO 82272 --- [rTaskExecutor-2] o.a.k.c.c.internals.AbstractCoordinator:[Consumer clientId = consumer-3,groupId = customer]尝试心跳失败,因为组正在重新平衡 2019-02-19 15:19:52.200 INFO 82272 --- [rTaskExecutor-1] o.a.k.c.c.internals.AbstractCoordinator:[Consumer clientId = consumer-2,groupId = customer]尝试心跳失败,因为组正在重新平衡 2019-02-19 15:19:52.200 INFO 82272 --- [rTaskExecutor-1] oakccinternals.ConsumerCoordinator:[Consumer clientId = consumer-2,groupId = customer]撤销先前分配的分区[spring-kafka-topic-2, spring-kafka-topic-0,spring-kafka-topic-1] 2019-02-19 15:19:52.200 INFO 82272 --- [rTaskExecutor-2] oakccinternals.ConsumerCoordinator:[Consumer clientId = consumer-3,groupId = customer]撤销先前分配的分区[spring-kafka-topic-4, spring-kafka-topic-5,spring-kafka-topic-3] 2019-02-19 15:19:52.200信息82272 --- [rTaskExecutor-1] osklKafkaMessageListenerContainer:分区被撤销:[spring-kafka-topic-2,spring-kafka-topic-0,spring-kafka-topic-1 ] 2019-02-19 15:19:52.200 INFO 82272 --- [rTaskExecutor-2] osklKafkaMessageListenerContainer:分区被撤销:[spring-kafka-topic-4,spring-kafka-topic-5,spring-kafka-topic-3 ] 2019-02-19 15:19:52.200 INFO 82272 --- [rTaskExecutor-1] o.a.k.c.c.internals.AbstractCoordinator:[Consumer clientId = consumer-2,groupId = customer](重新)加入组 2019-02-19 15:19:52.200 INFO 82272 --- [rTaskExecutor-2] o.a.k.c.c.internals.AbstractCoordinator:[Consumer clientId = consumer-3,groupId = customer](重新)加入组 2019-02-19 15:19:52.209 INFO 82272 --- [rTaskExecutor-1] o.a.k.c.c.internals.AbstractCoordinator:[Consumer clientId = consumer-2,groupId = customer]成功加入第582代的组 2019-02-19 15:19:52.209 INFO 82272 --- [rTaskExecutor-2] o.a.k.c.c.internals.AbstractCoordinator:[Consumer clientId = consumer-3,groupId = customer]成功加入第582代的组 2019-02-19 15:19:52.209信息82272 --- [rTaskExecutor-3] o.a.k.c.c.internals.AbstractCoordinator:[Consumer clientId = consumer-4,groupId = customer]成功加入第582代的组 2019-02-19 15:19:52.209信息82272 --- [rTaskExecutor-3] oakccinternals.ConsumerCoordinator:[Consumer clientId = consumer-4,groupId = customer]设置新分配的分区[spring-kafka-topic-4, spring-kafka-topic-5] 2019-02-19 15:19:52.210信息82272 --- [rTaskExecutor-1] oakccinternals.ConsumerCoordinator:[Consumer clientId = consumer-2,groupId = customer]设置新分配的分区[spring-kafka-topic-0, spring-kafka-topic-1] 2019-02-19 15:19:52.210信息82272 --- [rTaskExecutor-2] oakccinternals.ConsumerCoordinator:[Consumer clientId = consumer-3,groupId = customer]设置新分配的分区[spring-kafka-topic-2, spring-kafka-topic-3] 2019-02-19 15:19:52.211信息82272-[rTaskExecutor-3] o.s.k.l.KafkaMessageListenerContainer:分配的分区:[spring-kafka-topic-4,spring-kafka-topic-5] 2019-02-19 15:19:52.212信息82272 --- [rTaskExecutor-1] o.s.k.l.KafkaMessageListenerContainer:分配的分区:[spring-kafka-topic-0,spring-kafka-topic-1] 2019-02-19 15:19:52.212信息82272-[rTaskExecutor-2] o.s.k.l.KafkaMessageListenerContainer:分配的分区:[spring-kafka-topic-2,spring-kafka-topic-3] 消费者[customerTaskExecutor-3]收到消息[顾客(名称=测试6,电话号码= 6)]

1 个答案:

答案 0 :(得分:0)

阅读Kafka文档。

暂停使用者仅意味着随后的poll()不会返回任何记录,直到您调用resume()为止,但是您仍然必须在poll()内调用max.poll.interval.ms才能防止重新平衡。