我们正在运行一个独立的Apache Spark作业，该作业从MongoDB和HBase获取数据以生成数据段。由于以下错误，我们的Spark作业在中间终止-
java.io.FileNotFoundException: /var/log/listandclicker/blockmgr-e84681e4-9650-4042-803a-2c27b7d13cb1/0d/temp_shuffle_b06108fd-c766-445e-8f8c-e4dab5ccb245 (No such file or directory)
at java.io.FileOutputStream.open0(Native Method) ~[na:1.8.0_171]
at java.io.FileOutputStream.open(FileOutputStream.java:270) ~[na:1.8.0_171]
at java.io.FileOutputStream.<init>(FileOutputStream.java:213) ~[na:1.8.0_171]
at org.apache.spark.storage.DiskBlockObjectWriter$$anonfun$revertPartialWritesAndClose$2.apply$mcV$sp(DiskBlockObjectWriter.scala:215) ~[spark-core_2.11-2.2.0.jar!/:2.2.0]
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1346) [spark-core_2.11-2.2.0.jar!/:2.2.0]
at org.apache.spark.storage.DiskBlockObjectWriter.revertPartialWritesAndClose(DiskBlockObjectWriter.scala:212) [spark-core_2.11-2.2.0.jar!/:2.2.0]
at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.stop(BypassMergeSortShuffleWriter.java:237) [spark-core_2.11-2.2.0.jar!/:2.2.0]
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:102) [spark-core_2.11-2.2.0.jar!/:2.2.0]
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53) [spark-core_2.11-2.2.0.jar!/:2.2.0]
at org.apache.spark.scheduler.Task.run(Task.scala:108) [spark-core_2.11-2.2.0.jar!/:2.2.0]
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:335) [spark-core_2.11-2.2.0.jar!/:2.2.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_171]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_171]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_171]
任何见解或建议将不胜感激！非常感谢！