使Reactive Kafka与Confluent Schema Registry Avro Schema一起使用

Question

如何让Reactive Kafka（https://github.com/akka/reactive-kafka）与Confluent Schema Registry Avro Schema一起使用？这是我的示例代码：

def create(groupId: String)(implicit system: ActorSystem): Source[ConsumerMessage.CommittableMessage[String, Array[Byte]], Consumer.Control] = {
    
    val consumerSettings = ConsumerSettings(system, new StringDeserializer, new ByteArrayDeserializer)
      .withBootstrapServers(bootstrap)
      .withGroupId(groupId)
      .withProperty(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest")
      .withProperty("schema.registry.url", schemaRegistryUrl)
      .withProperty(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, classOf[String].getName)
      .withProperty(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, classOf[KafkaAvroDeserializer].getName)
      .withProperty(KafkaAvroDeserializerConfig.SPECIFIC_AVRO_READER_CONFIG, "true")
    
    Consumer.committableSource(consumerSettings, Subscriptions.topics(createDataSetJobRequestTopic))
}

Answer 1

您只需要将消费者配置为使用Confluent反序列化器：io.confluent.kafka.serializers.KafkaAvroDeserializer.class

设置以下属性：

• 的 ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG
• 的 ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG

此外，您必须添加属性＆＃34; schema.registry.url ＆＃34;为了指定至少一个指向您的架构注册表实例的地址。

最后，您必须在项目中添加以下依赖项：

            <dependency>
            <groupId>io.confluent</groupId>
            <artifactId>kafka-avro-serializer</artifactId>
            <version>${io.confluent.version}</version>
            <exclusions>
                <exclusion>
                    <groupId>org.slf4j</groupId>
                    <artifactId>slf4j-log4j12</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

Confluent Documentation