Question

通过手动kafka确认，在消费/处理大量消息时，我们观察到以下错误。

如果在创建使用者时将max.poll.records属性设置为1，那么在处理大量消息时会不会出现性能问题？？

2019-10-22 23:01:55.208 UTC [org.springframework.kafka.KafkaListenerEndpointContainer#0-2-C-1] ERROR c.d.s.s.b.a.kafka.KafkaRxTx - Kafka Listner Exception : Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records. -> {}
org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records.
    at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.sendOffsetCommitRequest(ConsumerCoordinator.java:808)
    at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.commitOffsetsSync(ConsumerCoordinator.java:691)
    at org.apache.kafka.clients.consumer.KafkaConsumer.commitSync(KafkaConsumer.java:1416)
    at org.apache.kafka.clients.consumer.KafkaConsumer.commitSync(KafkaConsumer.java:1377)
    at brave.kafka.clients.TracingConsumer.commitSync(TracingConsumer.java:151)
    at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.ackImmediate(KafkaMessageListenerContainer.java:922)
    at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.processAck(KafkaMessageListenerContainer.java:904)
    at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.access$2000(KafkaMessageListenerContainer.java:384)
    at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer$ConsumerAcknowledgment.acknowledge(KafkaMessageListenerContainer.java:1593)

Answer 1

是的，它将对性能产生重大影响；多少将取决于您的情况；您应该运行测试。

另外，考虑增加max.poll.interval.ms。

Kafka Listner异常：提交无法完成

1 个答案: