如何测试使用Avro和Confluent Schema Registry的Spring Cloud Stream Kafka Streams应用程序?

时间:2019-07-31 18:11:14

标签: apache-kafka spring-cloud apache-kafka-streams spring-kafka spring-cloud-stream

我在弄清楚如何测试使用Avro作为消息格式和(融合)架构注册表的Spring Cloud Stream Kafka Streams应用程序时遇到麻烦。

配置可能是这样的:

spring:
  application:
    name: shipping-service
  cloud:
    stream:
      schema-registry-client:
        endpoint: http://localhost:8081
      kafka:
        streams:
          binder:
            configuration:
              application:
                id: shipping-service
              default:
                key:
                  serde: org.apache.kafka.common.serialization.Serdes$IntegerSerde
              schema:
                registry:
                  url: ${spring.cloud.stream.schema-registry-client.endpoint}
              value:
                subject:
                  name:
                    strategy: io.confluent.kafka.serializers.subject.RecordNameStrategy
          bindings:
            input:
              consumer:
                valueSerde: io.confluent.kafka.streams.serdes.avro.SpecificAvroSerde
            order:
              consumer:
                valueSerde: io.confluent.kafka.streams.serdes.avro.GenericAvroSerde
            output:
              producer:
                valueSerde: io.confluent.kafka.streams.serdes.avro.SpecificAvroSerde
      bindings:
        input:
          destination: customer
        order:
          destination: order
        output:
          destination: order

server:
  port: 8086

logging:
  level:
    org.springframework.kafka.config: debug

注释:

  • 它正在使用本机序列化/反序列化。
  • 测试框架:Junit 5

我想关于Kafka Broker,我应该使用EmbeddedKafkaBroker bean,但是正如您所看到的,它还依赖于应该以某种方式模拟的Schema Registry。怎么样?

1 个答案:

答案 0 :(得分:2)

解决这个问题确实很痛苦,但最终我设法使用fluent-kafka-streams-tests使它起作用:

其他依赖项:

.class

关键是将必要的配置设置为系统属性。为此,我创建了一个单独的测试配置类:

testImplementation("org.springframework.kafka:spring-kafka-test")
testImplementation("com.bakdata.fluent-kafka-streams-tests:schema-registry-mock-junit5:2.0.0")

最后是测试类,您现在可以在其中使用KStream处理和利用模拟架构注册表来生成和使用Avro消息:

@Configuration
class KafkaTestConfiguration(private val embeddedKafkaBroker: EmbeddedKafkaBroker) {

    private val schemaRegistryMock = SchemaRegistryMock()

    @PostConstruct
    fun init() {
        System.setProperty("spring.kafka.bootstrap-servers", embeddedKafkaBroker.brokersAsString)
        System.setProperty("spring.cloud.stream.kafka.streams.binder.brokers", embeddedKafkaBroker.brokersAsString)
        schemaRegistryMock.start()
        System.setProperty("spring.cloud.stream.schema-registry-client.endpoint", schemaRegistryMock.url)
        System.setProperty("spring.cloud.stream.kafka.streams.binder.configuration.schema.registry.url", schemaRegistryMock.url)
    }

    @Bean
    fun schemaRegistryMock(): SchemaRegistryMock {
        return schemaRegistryMock
    }

    @PreDestroy
    fun preDestroy() {
        schemaRegistryMock.stop()
    }
}