Question

我正在尝试针对Apache Hadoop基本群集的BigInsights执行spark oozie example上的oozie_spark branch。

workflow.xml如下所示：

<workflow-app xmlns='uri:oozie:workflow:0.5' name='SparkWordCount'>
 <start to='spark-node' />
  <action name='spark-node'>
   <spark xmlns="uri:oozie:spark-action:0.1">
    <job-tracker>${jobTracker}</job-tracker>
    <name-node>${nameNode}</name-node>
    <master>${master}</master>
    <name>Spark-Wordcount</name>
    <class>org.apache.spark.examples.WordCount</class>
    <jar>/iop/apps/4.2.0.0/spark/jars/spark-assembly.jar,${jobDir}/lib/spark-wordcount-example.jar</jar>
    <spark-opts>--conf spark.driver.extraJavaOptions=-Diop.version=4.2.0.0</spark-opts>
    <arg>${inputDir}/FILE</arg>
    <arg>${outputDir}</arg>
    <capture-output/>
   </spark>
   <ok to="end" />
   <error to="fail" />
  </action>
  <kill name="fail">
   <message>Workflow failed, error
    message[${wf:errorMessage(wf:lastErrorNode())}]
   </message>
  </kill>
 <end name='end' />
</workflow-app>

configuration.xml：

<configuration>
    <property>
        <name>master</name>
        <value>local</value>
    </property>
    <property>
        <name>queueName</name>
        <value>default</value>
    </property>
    <property>
        <name>user.name</name>
        <value>default</value>
    </property>
    <property>
        <name>nameNode</name>
        <value>default</value>
    </property>
    <property>
        <name>jobTracker</name>
        <value>default</value>
    </property>
    <property>
        <name>jobDir</name>
        <value>/user/snowch/test</value>
    </property>
    <property>
        <name>inputDir</name>
        <value>/user/snowch/test/input</value>
    </property>
    <property>
        <name>outputDir</name>
        <value>/user/snowch/test/output</value>
    </property>
    <property>
        <name>oozie.wf.application.path</name>
        <value>/user/snowch/test</value>
    </property>
</configuration>

但是，错误是：

Exception in thread "main" org.apache.hadoop.gateway.shell.HadoopException: 
  org.apache.hadoop.gateway.shell.ErrorResponse: HTTP/1.1 400 Bad Request

我做错了什么？

Answer 1

问题似乎是元素<capture-output/>。

我删除了这个，错误消失了。

HTTP / 1.1 400错误请求执行oozie spark作业

1 个答案: