在将kafka流写入主题时如何处理avro反序列化异常

时间:2018-02-02 00:35:57

标签: apache-kafka apache-kafka-streams

在将流写入主题

时,我看到了一些例外情况
Output:
    Exception in thread "StreamThread-1" 
    org.apache.kafka.streams.errors.StreamsException: Failed to deserialize 
    value for record. topic=input_topic, partition=4, offset=9048083
Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id 572

这是代码。 键为null(字符串),值为avroserde

    streamsConfiguration.put(StreamsConfig.KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass().getName());
    streamsConfiguration.put(StreamsConfig.VALUE_SERDE_CLASS_CONFIG, SpecificAvroSerde.class);

我正在使用特定的Avro serde。所以我给了架构注册表的端点

    final Map<String, String> serdeConfig = Collections.singletonMap(
            AbstractKafkaAvroSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, schemaRegistryUrl);
    final Serde<avroschema> avroserde = new SpecificAvroSerde<>();
    MasterSpinsSerde.configure(serdeConfig, false); // `false` for record values

阅读源流,如下所示

    final KStreamBuilder builder = new KStreamBuilder();
    final KStream<String, avroschema> feeds = builder.stream("input_topic");

    feeds.to(Serdes.String(), avroserde,"output_topic");
    return new KafkaStreams(builder, streamsConfiguration);

0 个答案:

没有答案