Question

当两个不同的消费群体使用一个主题时，偏移量将重置。

我正在使用Kafka版本0.10.1.0和Spring-Kafka版本2.2.4发行版。我在主题“ topic_X”中产生一条消息，该消息应由两个不同的消费者组“ consumerA”和“ consumerB”使用，每个消费者组均具有一个。

让我们说“ consumerB”关闭了，我已经在“ topic_X”上生成了100条消息，并且“ consumerA”已经在运行并消耗了所有消息。当我再次调高“ consumberB”时，偏移量设置为100，而不是从0开始。

我尝试通过设置auto-offset-reset：最早，但仍然无法正常工作。以下是我从控制台获得的日志。

当我启动“ consumberB”时，我不想重置偏移量怎么办？

2019-05-15 11:28:44.309  INFO 16152 --- [eted-Data-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-4, groupId=consumer_group] (Re-)joining group
2019-05-15 11:28:44.536  INFO 16152 --- [ment-Data-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-2, groupId=consumer_group] Successfully joined group with generation 25
2019-05-15 11:28:44.539  INFO 16152 --- [ment-Data-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-2, groupId=consumer_group] Setting newly assigned partitions [topic_x-0]
2019-05-15 11:28:44.788  INFO 16152 --- [ment-Data-0-C-1] o.s.k.l.KafkaMessageListenerContainer    : partitions assigned: [topic_x-0]
2019-05-15 11:28:45.906  INFO 16152 --- [ment-Data-0-C-1] o.a.k.c.consumer.internals.Fetcher       : [Consumer clientId=consumer-2, groupId=consumer_group] Fetch offset 5411 is out of range for partition topic_x-0, resetting offset
2019-05-15 11:28:46.187  INFO 16152 --- [ment-Data-0-C-1] o.a.k.c.consumer.internals.Fetcher       : [Consumer clientId=consumer-2, groupId=consumer_group] Resetting offset for partition topic_x-0 to offset 5651.
2019-05-15 11:28:47.864  INFO 16152 --- [ment-Data-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-2, groupId=consumer_group] Attempt to heartbeat failed since group is rebalancing
2019-05-15 11:28:48.142  INFO 16152 --- [ment-Data-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-2, groupId=consumer_group] Revoking previously assigned partitions [topic_x-0]
2019-05-15 11:28:48.142  INFO 16152 --- [ment-Data-0-C-1] o.s.k.l.KafkaMessageListenerContainer    : partitions revoked: [topic_x-0]
2019-05-15 11:28:48.142  INFO 16152 --- [ment-Data-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-2, groupId=consumer_group] (Re-)joining group
2019-05-15 11:28:48.976  INFO 16152 --- [eted-Data-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-4, groupId=consumer_group] Successfully joined group with generation 26
2019-05-15 11:28:48.976  INFO 16152 --- [ment-Data-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-2, groupId=consumer_group] Successfully joined group with generation 26
2019-05-15 11:28:48.976  INFO 16152 --- [ment-Data-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-2, groupId=consumer_group] Setting newly assigned partitions [topic_x-0]
2019-05-15 11:28:48.976  INFO 16152 --- [eted-Data-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-4, groupId=consumer_group] Setting newly assigned partitions [topic_y-0]
2019-05-15 11:28:49.246  INFO 16152 --- [ment-Data-0-C-1] o.s.k.l.KafkaMessageListenerContainer    : partitions assigned: [topic_x-0]

下面是我用来创建使用者的代码。组“ consumerA”中的另一个使用者已经在不同的应用程序中运行。当我在“ consumerB”组中启动我的应用程序使用者时，偏移量将重置。

@KafkaListener(id="client-1", topics= "topic_x", groupId = "consumerB")
    public void completed(final byte[] bytes) throws IOException {

        //Handler code

    }

超出范围的偏移量会重置第2个消费群体的偏移量

0 个答案: