Question

我有一个接受GenericRecord类的发布者。

candle.exe : error CNDL0103 : The system cannot find the file 'Product.wxs' with type 'Source'.

我有以下avro架构：

@Override
public Future<RecordMetadata> publish(GenericRecord genericRecord) {
    Future<RecordMetadata> recordMetadataFuture =
            getPublisher().send(new ProducerRecord<>(producerConfiguration.getProperties()
                    .getProperty(ProducerConfiguration.PROPERTY_NAME_TOPIC), "sample.key",genericRecord));

    return recordMetadataFuture;
}

private KafkaProducer<String, GenericRecord> getPublisher() {
    return new KafkaProducer<>(producerConfiguration.getProperties());
}

我已经构建了自己的序列化程序：

Date Serializer：

{
"type" : "record",
"name" : "SampleDate",
"namespace": "com.sample.data.generated.avro",
"doc" : "sample date",
"fields" :  [
    {
        "name" : "sampleDate",
        "type" : {
            "type" : "int",
            "logicalType" : "date"
        }
    }
  ]
}

Generic Serializer：

@Component
public class SampleDateSerializer implements Serializer<GenericRecord> {

private AvroGenericSerializer serializer;

@Override
public void configure(Map<String, ?> configs, boolean isKey) {
    serializer = new AvroGenericSerializer(SampleDate.SCHEMA$);
}

@Override
public byte[] serialize(String topic, GenericRecord data) {
    return serializer.serialize(data);
}

@Override
public void close() {

}

但是，在测试我的发布者类时，我遇到以下错误：

public class AvroGenericSerializer {
private EncoderFactory avroEncoderFactory;
private DecoderFactory avroDecoderFactory;
private GenericDatumWriter<GenericRecord> avroWriter;
private GenericDatumReader<GenericRecord> avroReader;

public AvroGenericSerializer(Schema schema) {
    avroEncoderFactory = EncoderFactory.get();
    avroDecoderFactory = DecoderFactory.get();
    avroWriter = new GenericDatumWriter<>(schema);
    avroReader = new GenericDatumReader<>(schema);
}

public byte[] serialize(GenericRecord data) {
    final ByteArrayOutputStream stream = new ByteArrayOutputStream();
    final BinaryEncoder binaryEncoder = avroEncoderFactory.binaryEncoder(stream, null);
    try {
        avroWriter.write(data, binaryEncoder);
        binaryEncoder.flush();
        stream.close();
        return stream.toByteArray();
    } catch (IOException e) {
        throw new RuntimeException("Can't serialize Avro object", e);
    }
}

public GenericRecord deserialize(byte[] bytes) {
    try {
        return avroReader.read(null, avroDecoderFactory.binaryDecoder(bytes, null));
    } catch (IOException e) {
        throw new RuntimeException("Can't deserialize Avro object", e);
    }
 }
}

调试代码，我发现了

org.apache.kafka.common.errors.SerializationException: Can't convert value of class com.sample.data.generated.avro.SampleDate to class com.sample.message.serialize.SampleDateSerializer specified in value.serializer

方法在调用

时返回null

GenericDatumWriter.write()...

从

调用

Conversion conversion = this.getData().getConversionByClass(datum.getClass(), logicalType);

在这方面，我有没有办法填充

    org.apache.avro.generic.GenericData

    public <T> Conversion<T> getConversionByClass(Class<T> datumClass, LogicalType logicalType) {
    Map conversions = (Map)this.conversionsByClass.get(datumClass);
    return conversions != null?(Conversion)conversions.get(logicalType.getName()):null;
}

映射，以便它可以返回正确的转换器以用于给定的

 GenericData.conversionsByClass

Answer 1

我通过在GenericDatumWriter中传递GenericData对象来解决它。

My Generic Serializer现在看起来像这样：

    public AvroGenericSerializer(Schema schema) {
    avroEncoderFactory = EncoderFactory.get();
    avroDecoderFactory = DecoderFactory.get();
    final GenericData genericData = new GenericData();
    genericData.addLogicalTypeConversion(new TimeConversions.DateConversion());
    avroWriter = new GenericDatumWriter<>(schema, genericData);
    avroReader = new GenericDatumReader<>(schema);
}

Avro Date对象的SerializationException（Date LogicalType）

1 个答案: