Question

我正在尝试执行pig oozie工作流程。但是工作流程在运行状态下挂起，我检查了日志文件，我发现了这个

来自节点管理器的日志文件：

2015-02-25 17:50:06,322 [JobControl] INFO       org.apache.hadoop.yarn.client.api.impl.YarnClientImpl  - Submitted application application_1424690952568_0091 to ResourceManager at localhost/127.0.0.1:9003
2015-02-25 17:50:06,395 [JobControl] INFO    org.apache.hadoop.mapreduce.Job  - The url to track the job: http://localhost:8088/proxy/application_1424690952568_0091/
2015-02-25 17:50:06,396 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - HadoopJobId: job_1424690952568_0091
2015-02-25 17:50:06,396 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - Processing aliases a
2015-02-25 17:50:06,396 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - detailed locations: M: a[1,4] C:  R: 
2015-02-25 17:50:06,396 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - More information at: http://localhost:50030/jobdetails.jsp?jobid=job_1424690952568_0091
2015-02-25 17:50:06,456 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - 0% complete
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat

然后继续

This is my workflow.xml
<workflow-app xmlns="uri:oozie:workflow:0.2" name="pig-example">
<start to="pig-node"/>
<action name="pig-node">
 <pig>
        <job-tracker>localhost:9003</job-tracker>
        <name-node>hdfs://localhost:9000</name-node>
        <prepare> <delete path="hdfs://localhost:9000/pigout"/></prepare>
       <configuration>
            <property>
                <name>mapred.compress.map.output</name>
                <value>true</value>
            </property>
    <property>
                <name>mapred.job.queue.name</name>
                <value>${queueName}</value>
            </property>
           </configuration>
        <script>script.pig</script>
<param>input=${INPUT}</param>
 <param>ouput=${OUTPUT}</param>
   </pig>
   <ok to="end"/>
       <error to="fail"/>
    </action>
    <kill name="fail">
         <message>Pig failed, </message>
    </kill>
    <end name="end"/>

My job.properties 

nameNode=hdfs://localhost:9000
jobTracker=localhost:9003
queueName=default
oozie.libpath=/usr/lib/oozie-4.1.0/share/lib
oozie.use.system.libpath=true
oozie.wf.application.path=${nameNode}/pigoozie
INPUT=${nameNode}/a1
OUTPUT=${nameNode}/pigout

我不确定在hadoop 1中执行相同工作流程的问题是什么，它运行正常。我应该遵循任何额外的步骤在hadoop 2中运行oozie，如果是，请提及步骤

Answer 1

在workflow.xml或job.properties中设置以下属性。看起来您正在单个映射器上运行操作。 Oozie需要至少2个地图制作者。一个用于m / r启动器，一个用于实际操作。

mapred.tasktracker.map.tasks.maximum = 4和 mapred.tasktracker.reduce.tasks.maximum = 4

Oozie猪的工作流程在hadoop 2.2猪0.12.1 oozie 4.1.0

1 个答案: