在Spark集群环境中将检查点存储到EFS时遇到问题:
18/12/09 13:22:33错误CheckpointWriter:无法提交检查点 线程池执行者的任务 java.util.concurrent.RejectedExecutionException:任务 org.apache.spark.streaming.CheckpointWriter$CheckpointWriteHandler@4ed8ebc2 被拒绝 java.util.concurrent.ThreadPoolExecutor@252d7b55 [正在运行,池大小= 1,活动线程= 1,排队的任务= 1000,已完成的任务= 128] java.util.concurrent.ThreadPoolExecutor $ AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) 在 java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) 在 java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) 在 org.apache.spark.streaming.CheckpointWriter.write(Checkpoint.scala:290) 在 org.apache.spark.streaming.scheduler.JobGenerator.doCheckpoint(JobGenerator.scala:297) 在 org.apache.spark.streaming.scheduler.JobGenerator.org $ apache $ spark $ streaming $ scheduler $ JobGenerator $ processEvent(JobGenerator.scala:186) 在 org.apache.spark.streaming.scheduler.JobGenerator $ anon $ 1.onReceive(JobGenerator.scala:89) 在 org.apache.spark.streaming.scheduler.JobGenerator $ anon $ 1.onReceive(JobGenerator.scala:88) 在org.apache.spark.util.EventLoop $ anon $ 1.run(EventLoop.scala:48) 18/12/09 13:22:33错误CheckpointWriter:无法提交检查点 线程池执行者的任务 java.util.concurrent.RejectedExecutionException:任务 org.apache.spark.streaming.CheckpointWriter$CheckpointWriteHandler@44961d71 被拒绝 java.util.concurrent.ThreadPoolExecutor@252d7b55 [正在运行,池大小= 1,活动线程= 1,排队的任务= 1000,已完成的任务= 128] java.util.concurrent.ThreadPoolExecutor $ AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) 在 java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) 在 java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) 在 org.apache.spark.streaming.CheckpointWriter.write(Checkpoint.scala:290) 在 org.apache.spark.streaming.scheduler.JobGenerator.doCheckpoint(JobGenerator.scala:297) 在 org.apache.spark.streaming.scheduler.JobGenerator.org $ apache $ spark $ streaming $ scheduler $ JobGenerator $ processEvent(JobGenerator.scala:186) 在 org.apache.spark.streaming.scheduler.JobGenerator $ anon $ 1.onReceive(JobGenerator.scala:89) 在 org.apache.spark.streaming.scheduler.JobGenerator $ anon $ 1.onReceive(JobGenerator.scala:88) 在org.apache.spark.util.EventLoop $ anon $ 1.run(EventLoop.scala:48)处可以
任何人都可以帮助我解决该问题。预先感谢。