Question

我正在尝试运行wordcount map-reduce示例，引用https://support.pivotal.io/hc/en-us/articles/203355837-How-to-run-a-Map-Reduce-jar-using-Oozie-workflow。

我在群集设置中使用hadoop 2.2.0

问题是我的oozie map-reduce工作继续运行。它没有给出任何错误但是没有成功完成。没有错误消息。也没有创建输出目录（假设发生）。

下面是workflow.xml

    <workflow-app xmlns="uri:oozie:workflow:0.1" name="map-reduce-wf">
 <start to="mr-node"/>
 <action name="mr-node">
     <map-reduce>
       <job-tracker>${jobTracker}</job-tracker>
       <name-node>${nameNode}</name-node>
       <prepare>
         <delete path="${nameNode}/user/hadoop/${examplesRoot}/output-data/${outputDir}"/>
       </prepare>

   <configuration>
     <property>
       <name>mapred.mapper.new-api</name>
       <value>true</value>
     </property>
     <property>
       <name>mapred.reducer.new-api</name>
       <value>true</value>
     </property>
     <property>
       <name>mapred.job.queue.name</name>
       <value>${queueName}</value>
     </property>
     <property>
       <name>mapreduce.map.class</name>
       <value>org.apache.hadoop.examples.WordCount$TokenizerMapper</value>
     </property>
     <property>
       <name>mapreduce.reduce.class</name>
       <value>org.apache.hadoop.examples.WordCount$IntSumReducer</value>
     </property>
     <property>
       <name>mapreduce.combine.class</name>
       <value>org.apache.hadoop.examples.WordCount$IntSumReducer</value>
     </property>
     <property>
       <name>mapred.output.key.class</name>
       <value>org.apache.hadoop.io.Text</value>
     </property>
     <property>
       <name>mapred.output.value.class</name>
       <value>org.apache.hadoop.io.IntWritable</value>
     </property>
     <property>
       <name>mapred.input.dir</name>
       <value>/user/hadoop/${examplesRoot}/input-data/text</value>
     </property>
     <property>
       <name>mapred.output.dir</name>
       <value>/user/hadoop/${examplesRoot}/output-data/${outputDir}</value>
     </property>
   </configuration>
  </map-reduce>
  <ok to="end"/>
  <error to="fail"/>
 </action>
   <kill name="fail">
   <message>Map/Reduce failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
   </kill>
   <end name="end"/>
</workflow-app>

Job.properties

nameNode=hdfs://hadoopmaster:54310
jobTracker=localhost:54311
queueName=default
examplesRoot=examplesoozie

oozie.wf.application.path=${nameNode}/user/hadoop/${examplesRoot}/map-reduce
outputDir=map-reduce

作业状态的输出为

Workflow Name : map-reduce-wf
App Path      : hdfs://hadoopmaster:54310/user/hadoop/examplesoozie/map-reduce
Status        : RUNNING
Run           : 0
User          : hadoopuser
Group         : -
Created       : 2014-10-26 19:52 GMT
Started       : 2014-10-26 19:52 GMT
Last Modified : 2014-10-26 19:52 GMT
Ended         : -
CoordAction ID: -

Actions
------------------------------------------------------------------------------------------------------------------------------------
ID                                                                            Status    Ext ID                 Ext Status Err Code 
------------------------------------------------------------------------------------------------------------------------------------
0000006-141026135258981-oozie-hado-W@:start:                                  OK        -                      OK         -        
------------------------------------------------------------------------------------------------------------------------------------
0000006-141026135258981-oozie-hado-W@mr-node                                  PREP      -                      -          -        
------------------------------------------------------------------------------------------------------------------------------------

非常感谢任何帮助。

Answer 1

在job.properties中，initialDataset应该在jobStart（PST为GMT-8）之前8小时。您已经为两者设定了格林威治标准时间2014-10-26 19:52。

Oozie map-reduce工作永远停留在PREP状态

1 个答案: