Question

我正在尝试使用avro序列化，但是当我有多个要序列化的记录时，应用程序会挂起DataFileWriter close方法，但是它可以处理少量记录。

    final PipedOutputStream pipedOutputStream = new PipedOutputStream();
    PipedInputStream pipedInputStream = new PipedInputStream(
            pipedOutputStream);

    DatumWriter<DW> userDatumWriter = new SpecificDatumWriter<DW>(DW.class);
    DataFileWriter<DW> dataFileWriter = new DataFileWriter<DW>(
            userDatumWriter);

    dataFileWriter.create(payload.get(0).getSchema(), pipedOutputStream);

    for (DW currentRecord : payload) {
        dataFileWriter.append(currentRecord);
    }

    dataFileWriter.close();
    return pipedInputStream;

我一次添加10条记录后尝试刷新，但随后它会挂起刷新方法。

任何人都可以帮我吗？

Answer 1

通过返回ByteArrayOutputStream解决，如下所示：

    Schema schema = ReflectData.get().getSchema(DW.class);
    ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
    ReflectDatumWriter<Object> reflectDatumWriter = new ReflectDatumWriter<Object>(
            schema);
    DataFileWriter<Object> writer = new DataFileWriter<Object>(
            reflectDatumWriter).create(schema, outputStream);
    for (DW currentRecord : payload) {
        writer.append(currentRecord);
    }

    writer.close();
    return outputStream.toByteArray();

Avro序列化挂在DataFileWriter关闭方法上

1 个答案: