错误代码：JA018用于HDInsight spark2集群中的runnnig oozie工作流程

Question

我正在azure hdinsight spark2集群中安排具有以下结构的oozie作业。 我使用以下以下命令安排了作业，

oozie job -config /job.properties -run
oozie job -config /coordinator.properties -run

但我收到以下错误

Status: ERROR
Error Code: JA018
Error Message: Main class [org.apache.oozie.action.hadoop.ShellMain], exit code

我的workflow.xml文件：

<workflow-app name="sparkshellwf" xmlns="uri:oozie:workflow:0.3">
  <start to="sparkshellwf"/>
  <action name="sparkshellwf">
    <shell xmlns="uri:oozie:shell-action:0.1">
      <job-tracker>${jobTracker}</job-tracker>
      <name-node>${nameNode}</name-node>
      <configuration>
        <property>
          <name>mapred.job.queue.name</name>
          <value>${queueName}</value>
        </property>
      </configuration>
      <exec>$SPARK_HOME/bin/spark-submit</exec>
      <!--Adding all arguments needed/optional for Spark-submit here-->
      <argument>--class</argument>
      <argument>${Spark_Driver}</argument>
      <argument>--master</argument>
      <argument>${Spark_Master}</argument>
      <argument>--deploy-mode</argument>
      <argument>${Spark_Mode}</argument>
      <argument>--num-executors</argument>
      <argument>${numExecutors}</argument>
      <argument>--driver-memory</argument>
      <argument>${driverMemory}</argument>
      <argument>--executor-memory</argument>
      <argument>${executorMemory}</argument>
      <argument>--executor-cores</argument>
      <argument>${executorCores}</argument>
      <argument>${workflowRoot}/lib/${sparkJar}</argument>
    </shell>
    <ok to="end"/>
    <error to="fail"/>
  </action>
  <kill name="fail">
    <message>Job failed, error message[${wf:errorMessage(wf:lastErrorNode())}] </message>
  </kill>
  <end name="end"/>
</workflow-app>

但是Spark作业运行正常，没有任何错误

我得到的oozie回应如上所述。

Answer 1

JA018 Error is "output directory exists error" in workflow.
you can add following code to delete output directory in workfolw.xml.
<prepare>
            <delete path="[PATH_of_output directory]"/>
            ...
            <mkdir path="[PATH]"/>
            ...
</prepare>
you will get full log using following command

  oozie job -oozie http://hostname:11000/oozie -log job_id
  or
  yarn logs -applicationId <application_ID>
  You can refer link given below for different error codes
  https://oozie.apache.org/docs/4.2.0/oozie-default.xml