向Avro Schema中添加新字段获得兼容性异常

时间:2018-10-16 23:28:44

标签: java apache-kafka schema avro

我有一个简单的avro模式,并通过提供如下默认值将一个新字段添加到avro模式中:

{
    "name": "channelType",
    "type": [
        "null",
        {
            "type": "string",
            "avro.java.string": "String"
        }
    ],
    "default": null
},

使用以下代码反序列化数据。

public class RLoggerKafkaMessageDeSerializer implements Deserializer<RLoggerMessage> {
    private final DecoderFactory decoderFactory = DecoderFactory.get();
    private final DatumReader<RLoggerMessage> reader;

    public RLoggerKafkaMessageDeSerializer() {
        super();
        reader = new SpecificDatumReader<RLoggerMessage>(RLoggerMessage.getClassSchema());
    }

    @Override
    public RLoggerMessage deserialize(final String topic, final byte[] data) {
        try {
            BinaryDecoder decoder = decoderFactory.binaryDecoder(data, null);
            RLoggerMessage rlogMessage = reader.read(null, decoder);

            return rlogMessage;
        } catch (Exception e) {
            throw new SerializationException("Error deserialize message", e);
        } 
    }
}

然后我遇到以下错误:

java.io.EOFException
at org.apache.avro.io.BinaryDecoder.ensureBounds(BinaryDecoder.java:473)
at org.apache.avro.io.BinaryDecoder.readInt(BinaryDecoder.java:128)
at org.apache.avro.io.BinaryDecoder.readIndex(BinaryDecoder.java:423)
at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:290)
at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:267)
at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:178)
at org.apache.avro.specific.SpecificDatumReader.readField(SpecificDatumReader.java:116)
at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:230)
at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:174)
at org.apache.avro.specific.SpecificDatumReader.readField(SpecificDatumReader.java:116)
at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:230)
at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:174)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:152)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:144)

注意:我找到了一个解决方案,该解决方案是将旧模式和新模式都添加到SpecificDatumReader中。请参考以下链接:Avro schema doesn't honor backward compatibilty。 此解决方案不适用于我。我们的团队正在使用架构注册表。我已经将新架构上载到架构注册表。我了解是应该采用最新的架构来对数据进行序列化/反序列化。但是,出现以上异常。

0 个答案:

没有答案