我正在通过Flink仪表板向具有三个任务管理器的群集提交Flink作业。当我将Parallelism
设置为1(默认值)时,一切都按预期运行。但是,当我将Parallelism
增加到大于1的任何值时,作业将失败并带有例外,
java.io.FileNotFoundException: /tmp/flink-io-f91d7812-a411-4b58-a891-c9be1cde91da/08caeac37d6b8351daf6a3eb123a473106c56381b101f3e5d9704df9f78406a2.0.buffer (No such file or directory)
at java.io.RandomAccessFile.open0(Native Method)
at java.io.RandomAccessFile.open(RandomAccessFile.java:316)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243)
at org.apache.flink.streaming.runtime.io.BufferSpiller.createSpillingChannel(BufferSpiller.java:259)
at org.apache.flink.streaming.runtime.io.BufferSpiller.<init>(BufferSpiller.java:120)
at org.apache.flink.streaming.runtime.io.BarrierBuffer.<init>(BarrierBuffer.java:149)
at org.apache.flink.streaming.runtime.io.StreamInputProcessor.<init>(StreamInputProcessor.java:129)
at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.init(OneInputStreamTask.java:56)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:235)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
at java.lang.Thread.run(Thread.java:745)
我已经在我的Flink作业上启用了检查点,每10秒就有一次“完全一次”重试策略。这是我的检查点配置,
env.setStateBackend(new RocksDBStateBackend(props.getFlinkCheckpointDataUri(), true));
env.enableCheckpointing(10000, EXACTLY_ONCE); //10 seconds
CheckpointConfig config = env.getCheckpointConfig();
config.enableExternalizedCheckpoints(CheckpointConfig.ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION);
除了在仪表板中提交作业时输入2
之外,我是否应该配置其他内容?
编辑:如果我禁用检查点并上传作业,它将运行而不会出现任何错误。