当我第一次点击consumer.poll(1000)
时,我发现要花4-5分钟以上的时间才能返回消费者记录。有人知道为什么会这样吗?
此外,请注意,我已将auto-offset-reset-Config
属性设置为latest
,其中max-no-records
属性设置为1000
。
这确实引起了主要问题,因为我的应用程序是多线程的。请任何建议都会有所帮助
Consumer<String, String> consumer = ConsumerCreator.createConsumer();
int totalPollTimeMs = 0;
int pollTimems = 1000;
while (totalPollTimeMs < 5000) {
ConsumerRecords<String, String> consumerRecords = consumer.poll(pollTimems);
totalPollTimeMs += pollTimems;
if (consumerRecords.count() > 0) {
for (ConsumerRecord<String, String> consumerRecord : consumerRecords) {
if (null != consumerRecord.value()) {
try {
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
OutBoundRequest outBoundReq = objectMapper.readValue(consumerRecord.value(),OutBoundRequest.class);
ourBoundReqList.add(outBoundReq);
} catch (JsonParseException e) {
} catch (JsonMappingException e) {
} catch (IOException e) {
}
}
}
}
}
props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, kafkaBrokerParam.getValue());
props.put(ConsumerConfig.GROUP_ID_CONFIG, kafkaGroupIdparam.getValue());
props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
props.put(ConsumerConfig.MAX_POLL_RECORDS_CONFIG, 1000);
props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "true");
props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "latest");
Consumer<String, String> consumer = new KafkaConsumer<String, String>(props);
consumer.subscribe(Arrays.asList(kafkaTopicparam.getValue()));