Question

我有一个Java使用者，它可以连续运行大量数据和消息。有时，该程序会卡在消费者上。轮询数据..似乎是死锁或无限循环，但我不确定。尽管没有轮询实际数据，但没有引发异常，该程序似乎可以正常运行。（该问题仅发生在Java程序中。我运行控制台使用者，效果很好）

修复它的唯一方法是重新启动程序...

public class Main {
Properties consumerComfig = new Properties ();
consumerConfig.put(ConsumerConfig. KEY_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.ByteArrayDeserializer");
consumerConfig.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,"Org.apache.kafka.comnect.json.JsonDeserializer");
consumerConfig.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "some server")
consumerConfig.put(ConsumerConfig.GROUP_ID_CONFIG, UUID.randomUUID());
consumerConfig.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "false"):
consumerConfig.put(ConsumerConfig.SESSION_TIMEOUT_MS_CONFIG, 30000);
consumerConfig.put(ConsumerConfig.HEARTBEAT_INTERVAL_MS_CONFIG,10000)
consumerConfig.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "latest"):
consumer = new KafkaConsumer<String, JsonNode> (consumerConfig);

public void consumeAndSendToCorrelator () {
    logger.info ("Starting kafka consumer, listening to topics stringBuilder.toString ()");
    try {
        consumer.subscribe("INPUT TOPICS");
        while (true) {
            ConsumerRecords<String, JsonNode> records = consumer.poll(8000);
            logger.debug("Trying to pull records from consumer");
            JsonNode value = null;
            for (Consumer Record<String, JsonNode > irecord :records){
                try {
                    value = record.value();
                    if (value == null)
                        continue;
                    //Somme thread that process the message
                    correlatorActor.tell(value, ActorRef.noSender);
                } catch (JsonProcessingException e) {
                    logger.error("Could not parse incoming json" + value.toString());
                } catch (IOException e) {
                    logger.error("IOException occurred", e);
                } catch (Exception e) {
                    logger.error("Error occurred", e);
                }
                consumer.commitSync();
            }
        }
    }catch(WakeupException e) {
        logger.info("closed consumer");
    }finally {
        consumer.close()
    }
}

}

我使用Kafka 0.10 你知道会发生什么吗？我该如何解决？

卡夫卡民意调查突然陷入困境

0 个答案: