仅当我使用5000ms轮询时才出现Kafka轮询消息

Question

我有一个带有Spring-Kafka和Native Kafka代码块的应用程序。

简短逻辑：我的Spring Kafka Listener收到消息，在单独的线程中启动气流，代码等待直到完成气流任务，然后检查第二个主题中的消息并将结果消息发送到某个系统。

当我使用任何值而不是5000ms时，我的消费者无法分配该分区。任何大于5000ms的值都可以正常工作。除了kafka服务器版本外，我的devops工程师没有给我任何有关conf的信息。

我的代码 卡夫卡人

  private List<ExecutionTaskResultMessageInfo> getAllTransformationStatusEvents(Consumer<String, ExecutionTaskResult> consumer, String topic) {
        synchronized (this.consumer) {
            if (!isInited.get()) {
                consumer.subscribe(Collections.singletonList(topic));
                consumer.poll(Duration.ofMillis(0));
                this.isInited.set(true);
            }
            List<ExecutionTaskResultMessageInfo> events = new ArrayList<>();
            try {
                ConsumerRecords<String, ExecutionTaskResult> records = ConsumerRecords.empty();
                boolean empty = false;
                LOG.info("Starting to poll messages from transformation.status topic");
                while (!empty) {
                    try {
                        records = consumer.poll(Duration.ofMillis(kafkaConf.getPollTimeout()));
                    } catch (SerializationException e) {
                        LOG.error(e.getMessage());
                    }
                    if (!records.isEmpty()) {
                        for (ConsumerRecord<String, ExecutionTaskResult> record : records) {
                            ExecutionTaskResult executionTaskResult = record.value();
                            ExecutionTaskResultMessageInfo executionTaskResultMessageInfo;
                            if (isExecutionTaskResultEmpty(executionTaskResult)) {
                                executionTaskResultMessageInfo = new ExecutionTaskResultMessageInfo(executionTaskResult, record.offset(), record.partition(), true);
                                CustomEndPoint.Feature.incrementMapStatusErrorCount();
                                CustomEndPoint.Feature.addTransErrOffset(record.partition(), record.offset());
                            } else {
                                executionTaskResultMessageInfo = new ExecutionTaskResultMessageInfo(executionTaskResult, record.offset(), record.partition(), false);
                            }
                            int partition = executionTaskResultMessageInfo.getPartition();
                            kafkaNativeAcknowledgeObject.putMessageInfo(executionTaskResultMessageInfo, partition);
                            events.add(executionTaskResultMessageInfo);
                        }
                    } else {
                        empty = true;
                    }
                }
            } catch (Exception e) {
                LOG.info(e.toString());
            }
            LOG.info("Events from transformations.status topic: {}", events);
            LOG.info("Events collection size: {}", events.size());
            return events;
        }
    }

春天卡夫卡

 @KafkaListener(topics = "${kafka.execution-task-topic}", containerFactory = "executionTaskObjectContainerFactory")
    public void protocolEventsHandle(ExecutionTask executionTask,
                                     Acknowledgment acknowledgment,
                                     @Header(KafkaHeaders.RECEIVED_PARTITION_ID) int partition,
                                     @Header(KafkaHeaders.RECEIVED_TOPIC) String topic,
                                     @Header(KafkaHeaders.OFFSET) long offset) {

        ExecutionTaskMessageInfo messageInfo;
        if (isExecutionTaskEmpty(executionTask)) {
            LOG.info("Getting new empty message from offset: {} from partition: {} and topic: {}", offset, partition, topic);
            messageInfo = new ExecutionTaskMessageInfo(offset, partition, true, acknowledgment);
            kafkaAcknowledgeObject.putMessageInfo(messageInfo, partition);
            CustomEndPoint.Feature.incrementBDCPErrorCount();
            CustomEndPoint.Feature.addBDCPErrOffset(partition, offset);
        } else {
            LOG.info("Getting new message offset: {} from partition: {} and topic: {}", offset, partition, topic);
            try {
                ResponseEntity<String> stringResponseEntity = airflowRestRunner.startDag(executionTask);
                String runId = getRunId(stringResponseEntity);
                messageInfo = new ExecutionTaskMessageInfo(offset, partition, false, acknowledgment);
                kafkaAcknowledgeObject.putMessageInfo(messageInfo, partition);
                this.executorService.submit(kafkaAlertProducer.produceMessageAfterTaskSuccess(runId, executionTask, messageInfo));
            } catch (Exception e) {
                LOG.info(e.toString());
                messageInfo = new ExecutionTaskMessageInfo(offset, partition, true, acknowledgment);
                kafkaAcknowledgeObject.putMessageInfo(messageInfo, partition);
            }
        }
    }

服务器kafka版本1.0.0 应用kafka版本2.0.1