无法从Apache Beam中的avro-parquet模式读取日期格式列(int96类型)

时间:2019-04-24 11:38:14

标签: apache-beam apache-beam-io

读取具有 date 列的镶木地板文件时,我面临以下异常。我正在使用 beam-sdks-java-io * 2.11.0和实木复合地板* -1.10 请帮我。

先谢谢您。

Caused by: java.lang.IllegalArgumentException: INT96 not yet implemented.
    at org.apache.parquet.avro.AvroSchemaConverter$1.convertINT96(AvroSchemaConverter.java:279)
    at org.apache.parquet.avro.AvroSchemaConverter$1.convertINT96(AvroSchemaConverter.java:264)
    at org.apache.parquet.schema.PrimitiveType$PrimitiveTypeName$7.convert(PrimitiveType.java:297)
    at org.apache.parquet.avro.AvroSchemaConverter.convertField(AvroSchemaConverter.java:263)
    at org.apache.parquet.avro.AvroSchemaConverter.convertFields(AvroSchemaConverter.java:241)
    at org.apache.parquet.avro.AvroSchemaConverter.convert(AvroSchemaConverter.java:231)
    at org.apache.parquet.avro.AvroReadSupport.prepareForRead(AvroReadSupport.java:130)
    at org.apache.parquet.hadoop.InternalParquetRecordReader.initialize(InternalParquetRecordReader.java:183)
    at org.apache.parquet.hadoop.ParquetReader.initReader(ParquetReader.java:156)
    at org.apache.parquet.hadoop.ParquetReader.read(ParquetReader.java:135)
    at org.apache.beam.sdk.io.parquet.ParquetIO$ReadFiles$ReadFn.processElement(ParquetIO.java:221)

1 个答案:

答案 0 :(得分:0)

根据 Parquet-Avro 文档,Avro 类型映射不支持 int96(实木复合地板类型)。