Question

当我尝试在flatMap运算符中写入OUT集合时，出现非法状态异常（仅在高负载下）：缓冲池被破坏 我在这里做什么错？当flink抛出缓冲池错误吗？

java.lang.RuntimeException:  Buffer pool is destroyed.

    at org.apache.flink.streaming.runtime.io.RecordWriterOutput.pushToRecordWriter(RecordWriterOutput.java:110)
    at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:89)
    at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:45)
    at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:718)
    at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:696)
    at org.apache.flink.streaming.api.operators.TimestampedCollector.collect(TimestampedCollector.java:51)

..........
Caused by: java.lang.IllegalStateException:  Buffer pool is destroyed.
    at org.apache.flink.util.Preconditions.checkState(Preconditions.java:179)
    at org.apache.flink.runtime.io.network.buffer.BufferBuilder.append(BufferBuilder.java:79)
    at org.apache.flink.runtime.io.network.api.serialization.SpanningRecordSerializer.copyToBufferBuilder(SpanningRecordSerializer.java:95)
    at org.apache.flink.runtime.io.network.api.writer.RecordWriter.copyFromSerializerToTargetChannel(RecordWriter.java:150)
    at org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:128)
    at org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:101)
    at org.apache.flink.streaming.runtime.io.StreamRecordWriter.emit(StreamRecordWriter.java:81)
    at org.apache.flink.streaming.runtime.io.RecordWriterOutput.pushToRecordWriter(RecordWriterOutput.java:107)```

Answer 1

什么是缓冲池？

缓冲池是用于网络堆栈的内存段实例的固定大小的池，这意味着您在发送数据时将其放入其中，并在通过网络接收时从中获取分段。

为什么要摧毁它？

由于网络环境关闭，缓冲池被破坏。

为什么NetworkEnvironment关闭？

这可能是具体情况，我不太确定。我建议您在此缓冲池例外之上阅读更多日志。

在Apache Flink flapMap Operator中发现“缓冲池被破坏”的问题

1 个答案: