Kafka Consumer对于多线程访问异常并不安全

时间:2018-08-23 08:19:04

标签: java apache-kafka kafka-consumer-api

当我使用kafka创建消费者时,出现以下异常:

例外:

  

Kafka Consumer对于多线程访问异常并不安全

同时有100多个消费者订阅主题,同时也获得了记录。

我添加了以下代码

//KafkaConsumer Configuration 

KafkaConsumer<String, Object> kafkaConsumer;
Properties properties = new Properties();
properties.put("bootstrap.servers", KAFKA_SERVER_URL + ":" + KAFKA_SERVER_PORT);
properties.put("group.id", GROUP_ID);
properties.put("enable.auto.commit", "false");
properties.put("auto.commit.interval.ms", "1000");
properties.put("auto.offset.reset", "earliest");
properties.put("key.deserializer", JsonDeserializer.class);
properties.put("value.deserializer", JsonDeserializer.class);
kafkaConsumer = new KafkaConsumer<>(properties);


//Subscribe Consumer

kafkaConsumer.subscribe(Arrays.asList("Topic_Name"));

//Get Records From Beginning

while (true) {
    kafkaConsumer.seekToBeginning(kafkaConsumer.assignment());
    ConsumerRecords<String, Object> records = kafkaConsumer.poll(Duration.ofMillis(1000));
    for (ConsumerRecord<String, Object> record : records) {
        System.out.printf("offset = %d, key = %s, value = %s%n", record.offset(), 
            record.key(), record.value());
    }
}

//Get Records Using WebSocket this section continuously

ConsumerRecords<String, Object> records = kafkaConsumer.poll(Duration.ofMillis(1000));
for (ConsumerRecord<String, Object> record : records) {
    System.out.printf("offset = %d, key = %s, value = %s%n", record.offset(), 
        record.key(), record.value());
}

0 个答案:

没有答案