Question

我有一个Kafka消费者，只订阅了一个主题。在某个时间点，正常工作后，我在日志中收到以下消息：

Line 25694: 2017-05-15 17:59:53.656 [INFO ] [MeasureConsumerExecutor] AbstractCoordinator - Attempt to heart beat failed since the group is rebalancing, try to re-join group.
Line 25739: 2017-05-15 18:01:39.745 [INFO ] [MeasureConsumerExecutor] AbstractCoordinator - Marking the coordinator 2147483647 dead.
Line 25740: 2017-05-15 18:01:39.745 [WARN ] [MeasureConsumerExecutor] ConsumerCoordinator - Auto offset commit failed: null
Line 25766: 2017-05-15 18:10:52.539 [INFO ] [MeasureConsumerExecutor] AbstractCoordinator - Marking the coordinator 2147483647 dead.
Line 25789: 2017-05-15 18:25:51.036 [INFO ] [MeasureConsumerExecutor] AbstractCoordinator - Marking the coordinator 2147483647 dead.
Line 25790: 2017-05-15 18:25:52.241 [WARN ] [MeasureConsumerExecutor] ConsumerCoordinator - Auto offset commit failed: null
Line 25796: 2017-05-15 18:31:10.354 [INFO ] [MeasureConsumerExecutor] AbstractCoordinator - Marking the coordinator 2147483647 dead.
Line 25797: 2017-05-15 18:31:24.101 [INFO ] [MeasureConsumerExecutor] EventConsumer - run() - WARN - msg: KafkaConsumer will be CLOSED!

我的代码很简单：

private final AtomicBoolean closed = new AtomicBoolean(false); ...
...
...
try {   
        while (!closed.get()) {
            ConsumerRecords<String, Message> records = kafkaConsumer.poll(Long.MAX_VALUE);
            for (ConsumerRecord<String, Message> record : records) {
                Message message = record.value();
                messageArrived(message);
            }
        }
        logger.info("run() - NOTIFY - msg: idConsumer = [{}] HAS !closed.get() = [{}]", consumerId, !closed.get());
    } catch (WakeupException wakeupException) {
        logger.error("run() - ERROR - msg: Error on Consumer [{}] caused by = [{}]", getConsumerId(), wakeupException.getMessage(), wakeupException);
        // Ignore exception if closing 
        if (!closed.get()) 
            throw wakeupException;
    } catch (KafkaException kafkaException) {
        logger.error("run() - ERROR - msg: Error on Consumer [" + getConsumerId() + "] caused by = [" + kafkaException.getMessage() + "]", kafkaException);
    } catch (Exception exception) {
        logger.error("run() - ERROR - msg: Error on Consumer [" + getConsumerId() + "] caused by = [" + exception.getMessage() + "]", exception);
    } finally {
        logger.info("run() - WARN - msg: KafkaConsumer will be CLOSED!");
        if (null != kafkaConsumer) {
            kafkaConsumer.close();
        }
    }
}

奇怪的是，我得到了最后一个WARN日志（“KafkaConsumer将被关闭”）而没有进入异常日志（显然没有例外）并且没有“封闭”变量无处可更改。

我有多个消费者在不同的主题上并行运行，但我认为这不相关。代理位于同一子网中的另一台物理计算机上。

你能不能给我一些关于这里发生了什么的提示，如何处理这个问题以防止消费者断线或至少能够从中恢复？

非常感谢。

Kafka使用者在提交偏移和重新平衡时失败

0 个答案: