在hue中使用oozie工作流执行MapReduce作业，给出错误的输出

Question

我正在尝试使用色调中的oozie工作流执行MapReduce作业。当我提交作业时，oozie成功执行但我没有得到预期的输出。似乎mapper或reducer都没有被调用过。我的workflow.xml是：

<workflow-app name="wordCount" xmlns="uri:oozie:workflow:0.4">
    <start to="wordcount"/>
    <action name="wordcount">
        <map-reduce>
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <configuration>
                <property>
                    <name>mapred.input.dir</name>
                    <value>/user/root/jane/inputPath</value>
                </property>
                <property>
                    <name>mapred.output.dir</name>
                    <value>/user/root/jane/outputPath17</value>
                </property>
                <property>
                    <name>mapred.mapper.class</name>
                    <value>MapReduceGenerateReports.Map</value>
                </property>
                <property>
                    <name>mapred.reducer.class</name>
                    <value>MapReduceGenerateReports.Reduce</value>
                </property>
                <property>
                    <name>mapred.mapper.new-api</name>
                    <value>true</value>
                </property>
                <property>
                    <name>mapred.reducer.new-api</name>
                    <value>true</value>
                </property>
            </configuration>
        </map-reduce>
        <ok to="end"/>
        <error to="kill"/>
    </action>
    <kill name="kill">
        <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>

任何人都可以告诉我这是什么问题吗？

我的新workflow.xml：

<workflow-app name="wordCount" xmlns="uri:oozie:workflow:0.4">
    <start to="wordcount"/>
    <action name="wordcount">
        <map-reduce>
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <configuration>
                <property>
                    <name>mapred.input.dir</name>
                    <value>/user/root/jane/inputPath</value>
                </property>
                <property>
                    <name>mapred.output.dir</name>
                    <value>/user/root/jane/outputPath3</value>
                </property>
                <property>
                    <name>mapred.mapper.new-api</name>
                    <value>true</value>
                </property>
                <property>
                    <name>mapred.reducer.new-api</name>
                    <value>true</value>
                </property>
                <property>
                    <name>mapreduce.map.class</name>
                    <value>MapReduceGenerateReports$Map</value>
                </property>
                <property>
                    <name>mapreduce.reduce.class</name>
                    <value>MapReduceGenerateReports$Reduce</value>
                </property>
                <property>
                    <name> mapred.output.key.class</name>
                    <value>org.apache.hadoop.io.LongWritable</value>
                </property>
                <property>
                    <name>mapred.output.value.class</name>
                    <value>org.apache.hadoop.io.Text</value>
                </property>
            </configuration>
        </map-reduce>
        <ok to="end"/>
        <error to="kill"/>
    </action>
    <kill name="kill">
        <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>

jobtracker日志：

1）

Kind    % Complete  Num Tasks   Pending Running Complete    Killed  Failed/Killed
Task Attempts
map 100.00%
1   0   0   1   0   0 / 0
reduce  100.00%
0   0   0   0   0   0 / 0

2）

   Kind Total Tasks(successful+failed+killed)   Successful tasks    Failed tasks    Killed tasks    Start Time  Finish Time
    Setup    1   1   0   0  5-Apr-2014 18:36:22 5-Apr-2014 18:36:23 (1sec)
    Map  1   1   0   0  5-Apr-2014 18:33:27 5-Apr-2014 18:33:33 (5sec)
    Reduce   0   0   0   0      
    Cleanup  1   1   0   0  5-Apr-2014 18:33:33 5-Apr-2014 18:33:37 (4sec)

Answer 1

查看有关使用新API的说明here

  

但是，如果您确实需要在Oozie中运行使用20 API编写的MapReduce作业，则需要在workflow.xml中进行更改。

     

  
1. 将mapred.mapper.class更改为 mapreduce.map.class
  
2. 将mapred.reducer.class更改为 mapreduce.reduce.class
  
3. 添加mapred.output.key.class
  
4. 添加mapred.output.value.class
  
5. 并且，将以下属性包含在MR操作配置中