我已经编写了shell脚本来运行spark作业:
/home/ajit/deviceAnalysis.sh
/usr/spark/bin/spark-submit --master yarn --deploy-mode cluster --packages org.mongodb.spark:mongo-spark-connector_2.11:2.4.1 --conf spark.yarn.maxAppAttempts=4 --conf spark.yarn.am.attemptFailuresValidityInterval=1h --class com.mmi.probes.analysis.app.StartApp --driver-memory 2g --queue deviceAnalysis ./DailyDeviceProbesAnalysis.jar
({{1)}可以正常工作。但是我想使用crontab每5分钟运行一次,但运行时却抛出异常:
sh /home/ajit/deviceAnalysis.sh
在crontab中
java.io.FileNotFoundException: File file:/home/ajit/.sparkStaging/application_1601983398886_0126/__spark_libs__9052532821082604838.zip does not exist
请帮我现在该怎么办