如何在Azkaban中运行火花作业?
我应该使用command=spark-submit...
还是其他的东西?
也许在azkaban
中有火花的工作类型?
目前,我只有一个spark-submiy
命令,并试图将其放入作业文件中。
但是我有错误:
java.lang.RuntimeException: azkaban.jobExecutor.utils.process.ProcessFailureException
at azkaban.jobExecutor.ProcessJob.run(ProcessJob.java:198)
at azkaban.execapp.JobRunner.runJob(JobRunner.java:692)
at azkaban.execapp.JobRunner.run(JobRunner.java:543)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: azkaban.jobExecutor.utils.process.ProcessFailureException
at azkaban.jobExecutor.utils.process.AzkabanProcess.run(AzkabanProcess.java:131)
at azkaban.jobExecutor.ProcessJob.run(ProcessJob.java:192)
... 7 more