Question

我正在尝试将Java中的JSON转换为镶木地板格式，但出现异常。

输入JSON：

{"list": [ {"mainBearingX": 0.178334,
            "gearBoxZ": 0.03885,
            "_t": 1560305236290000,
            "mainBearingZ": 0.034438,
            "gearBoxX": 0.035738,
            "mainBearingY": 0.029445,
            "gearBoxY": 0.040929,
            "generatorX": 0.776837,
            "generatorY": 0.124234,
            "ts_id":"t1"
           },
           {"mainBearingX": 0.169478,
            "gearBoxZ": 0.008242,
            "_t": 1560305236311000,
            "mainBearingZ": 0.007531,
            "gearBoxX": 0.025647,
            "mainBearingY": 0.029445,
            "gearBoxY": 0.026282,
            "generatorX": 0.770189,
            "generatorY": 0.117464,
            "ts_id": "t1"
           }
         ]
}

代码：

public static void toConvert(OutPut output) {
            String inputFile = "test.parquetFile";
            Path dataFile = new Path(inputFile);
            Schema schema = ReflectData.AllowNull.get().getSchema(OutPut.class);
            try (ParquetWriter<OutPut> writer = AvroParquetWriter.<OutPut>builder(dataFile)
                    .withSchema(schema)
                    .withDataModel(ReflectData.get())
                    .withConf(new Configuration())
                    .withCompressionCodec(CompressionCodecName.SNAPPY)
                    .withWriteMode(Mode.OVERWRITE)
                    .build()) {
            } catch (IOException e) {
                e.printStackTrace();
            }
public class OutPut {
    List<Map<String, Object>> list;
}

例外：

Exception in thread "main" org.apache.parquet.schema.InvalidSchemaException: Cannot write a schema with an empty group: required group value {}
at org.apache.parquet.schema.TypeUtil$1.visit(TypeUtil.java:27)
at org.apache.parquet.schema.GroupType.accept(GroupType.java:226)
at org.apache.parquet.schema.TypeUtil$1.visit(TypeUtil.java:31)
at org.apache.parquet.schema.GroupType.accept(GroupType.java:226)
at org.apache.parquet.schema.TypeUtil$1.visit(TypeUtil.java:31)
at org.apache.parquet.schema.GroupType.accept(GroupType.java:226)
at org.apache.parquet.schema.TypeUtil$1.visit(TypeUtil.java:31)
at org.apache.parquet.schema.GroupType.accept(GroupType.java:226)
at org.apache.parquet.schema.TypeUtil$1.visit(TypeUtil.java:31)
at org.apache.parquet.schema.TypeUtil$1.visit(TypeUtil.java:37)
at org.apache.parquet.schema.MessageType.accept(MessageType.java:55)
at org.apache.parquet.schema.TypeUtil.checkValidWriteSchema(TypeUtil.java:23)
at org.apache.parquet.hadoop.ParquetFileWriter.<init>(ParquetFileWriter.java:228)
at org.apache.parquet.hadoop.ParquetWriter.<init>(ParquetWriter.java:273)
at org.apache.parquet.hadoop.ParquetWriter$Builder.build(ParquetWriter.java:494)

Answer 1

问题在于您的OutPut类型包含类型Object作为Map中的值类型：

public class OutPut {
    List<Map<String, Object>> list;
}

您正在使用ReflectData通过内省为您的类型推断一个Avro模式。但是，它无法从Object类型中推断出任何有用的信息。

如果您将OutPut的定义更改为使用具体类型，例如：

public class OutPut {
    List<Map<String, Double>> list;
}

然后它应该工作。

将JSON转换为Java语言

1 个答案: