Question

我的Kafka制作人正在以avro格式发送数据，我正在阅读使用avro-schema（SpinsAvro）并尝试将流打印到控制台。

但是输出结果如下：Key为null，但值就像垃圾一样。

[KSTREAM-SOURCE-0000000000]: null , [B@200868f5
[KSTREAM-SOURCE-0000000000]: null , [B@841bc92
[KSTREAM-SOURCE-0000000000]: null , [B@302e9607
[KSTREAM-SOURCE-0000000000]: null , [B@6f9139fb
[KSTREAM-SOURCE-0000000000]: null , [B@cbdab3c

以下是代码：

public class TestStream {

public static void main(final String[] args) throws Exception {

    final String bootstrapServers = "kafka-XXX:9092";
    final String schemaRegistryUrl = "http://XXX:8081";
    final KafkaStreams streams = buildStream(
            bootstrapServers,
            schemaRegistryUrl,
            "/tmp/kafka-streams");

    streams.cleanUp();

    streams.start();

    // Add shutdown hook to respond to SIGTERM and gracefully close Kafka Streams
    Runtime.getRuntime().addShutdownHook(new Thread(streams::close));
}

static KafkaStreams buildStream(final String bootstrapServers,
                                   final String schemaRegistryUrl,
                                   final String stateDir) {
    final Properties streamsConfiguration = new Properties();

    streamsConfiguration.put(StreamsConfig.APPLICATION_ID_CONFIG, "lambda-example");
    streamsConfiguration.put(StreamsConfig.CLIENT_ID_CONFIG, "lambda-example-client");
    streamsConfiguration.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
    streamsConfiguration.put(AbstractKafkaAvroSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, schemaRegistryUrl);
    streamsConfiguration.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
    streamsConfiguration.put("value.deserializer", "io.confluent.kafka.serializers.KafkaAvroDeserializer");
    streamsConfiguration.put(StreamsConfig.STATE_DIR_CONFIG, stateDir);
    streamsConfiguration.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
    streamsConfiguration.put(StreamsConfig.COMMIT_INTERVAL_MS_CONFIG, 10 * 1000);


    final KStreamBuilder builder = new KStreamBuilder();

    // read the source stream
    final KStream<String, SpinsAvro> feeds = builder.stream("spins_topic");
    feeds.print();

    return new KafkaStreams(builder, streamsConfiguration);

}

}

Answer 1

您的配置不正确。

Kafka Streams使用Serdes组合序列化器和反序列化器，用于一个类中的单个类型。设置

streamsConfiguration.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
streamsConfiguration.put("value.deserializer", "io.confluent.kafka.serializers.KafkaAvroDeserializer");

没有任何影响。您需要使用DEFAULT_KEY_SERDE_CLASS_CONFIG和DEFAULT_VALUE_SERDE_CLASS_CONFIG并传入相应的Serdes。

原始类型的Serdes已经通过Serdes类提供（例如，Serdes.Long()）。您还可以使用类WrapperSerde来插入自定义序列化程序和反序列化程序来创建自定义Serde（或者您只是从头开始实现Serde接口）。

查看文档了解更多详情：https://kafka.apache.org/10/documentation/streams/developer-guide/datatypes.html

Kafka流不以avro格式打印值

1 个答案: