Question

在集成测试的情况下，我将记录发送到Kafka，我想知道何时处理和提交，然后执行我的断言（而不是使用Thread.sleep）... < / p>

这是我的尝试：

public void sendRecordAndWaitUntilItsNotConsumed(ProducerRecord<String, String> record)
      throws ExecutionException, InterruptedException {

    RecordMetadata recordMetadata = producer.send(record).get();
    TopicPartition topicPartition = new TopicPartition(recordMetadata.topic(),
        recordMetadata.partition());

    try (KafkaConsumer<String, String> consumer = new KafkaConsumer<>(consumerConfig)) {

      consumer.assign(Collections.singletonList(topicPartition));

      long recordOffset = recordMetadata.offset();
      long currentOffset = getCurrentOffset(consumer, topicPartition);

      while (currentOffset <= recordOffset) {
        currentOffset = getCurrentOffset(consumer, topicPartition);
        LOGGER.info("Waiting for message to be consumed - Current Offset = " + currentOffset
            + " - Record Offset = " + recordOffset);
      }
    }
  }

  private long getCurrentOffset(KafkaConsumer<String, String> consumer,
      TopicPartition topicPartition) {

    consumer.seekToEnd(Collections.emptyList());

    return consumer.position(topicPartition);
  }

但它不起作用。实际上，我禁用了消费者的提交，并且它不会在Waiting for message to be consumed...

上循环

Answer 1

使用KafkaConsumer#committed代替KafkaConsumer#position。

private void sendRecordAndWaitUntilItsNotConsumed(ProducerRecord<String, String> record) throws InterruptedException, ExecutionException {

        RecordMetadata recordMetadata = producer.send(record).get();

        TopicPartition topicPartition = new TopicPartition(recordMetadata.topic(),
                recordMetadata.partition());

        consumer.assign(Collections.singletonList(topicPartition));

        long recordOffset = recordMetadata.offset();
        long currentOffset = getCurrentOffset(topicPartition);

        while (currentOffset < recordOffset) {
            currentOffset = getCurrentOffset(topicPartition);
            LOGGER.info("Waiting for message to be consumed - Current Offset = " + currentOffset
                    + " - Record Offset = " + recordOffset);
            TimeUnit.MILLISECONDS.sleep(200);
        }
    }

    private long getCurrentOffset(TopicPartition topicPartition) {
        OffsetAndMetadata offsetAndMetadata = consumer.committed(topicPartition);
        return offsetAndMetadata != null ? offsetAndMetadata.offset() - 1 : -1;
    }

如何知道Kafka何时提交记录？

1 个答案: