"提交失败的抵消"同时提交偏移量

时间:2017-07-06 19:32:42

标签: java apache-kafka kafka-consumer-api

我有一个kafka消费者,我从中消耗特定主题的数据,我看到下面的异常。我正在使用0.10.0.0 kafka版本。

LoggingCommitCallback.onComplete: Commit failed for offsets= {....}, eventType= some_type, time taken= 19ms, error= org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured session.timeout.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records.

我添加了这两个额外的消费者属性,但它仍然没有帮助:

session.timeout.ms=20000
max.poll.records=500

我在不同的后台线程中提交偏移量,如下所示:

kafkaConsumer.commitAsync(new LoggingCommitCallback(consumerType.name()));

该错误意味着什么,我该如何解决?我是否需要添加其他一些消费者属性?

1 个答案:

答案 0 :(得分:0)

是的,降低max.poll.records。您将获得较小批量的数据,但更频繁的调查调用将有助于使会话保持活跃。