Question

我使用 Apache Oozie 4.3.0 的版本以及 Hadoop 2.7.3

我开发了一个非常简单的Oozie工作流，它只是有一个sqoop操作，可以将系统事件导出到MySQL表。

<workflow-app name="WorkflowWithSqoopAction" xmlns="uri:oozie:workflow:0.1">
    <start to="sqoopAction"/>
        <action name="sqoopAction">
                <sqoop xmlns="uri:oozie:sqoop-action:0.2">
                        <job-tracker>${jobTracker}</job-tracker>
                        <name-node>${nameNode}</name-node>
                        <command>export --connect jdbc:mysql://localhost/airawat --username devUser --password myPwd --table eventsgranularreport  --direct --enclosed-by '\"' --export-dir /user/hive/warehouse/eventsgranularreport </command>
                </sqoop>
                <ok to="end"/>
                <error to="killJob"/>
        </action>
    <kill name="killJob">
            <message>"Killed job due to error: ${wf:errorMessage(wf:lastErrorNode())}"</message>
        </kill>
    <end name="end" />
</workflow-app>

我在HDFS中部署的应用程序如下：

hdfs dfs -ls -R /oozieProject | awk '{ print $8 }'

/oozieProject/workflowSqoopAction
/oozieProject/workflowSqoopAction/README.md
/oozieProject/workflowSqoopAction/job.properties
/oozieProject/workflowSqoopAction/workflow.xml

hdfs dfs -ls -d /oozieProject

drwxr-xr-x   - sergio supergroup          0 2017-04-15 14:08 /oozieProject

我在 job.properties 中包含了以下配置：

#*****************************
# job.properties
#*****************************

nameNode=hdfs://localhost:9000
jobTracker=localhost:8032
queueName=default

mapreduce.job.user.name=sergio
user.name=sergio
oozie.libpath=${nameNode}/oozieProject/share/lib
oozie.use.system.libpath=true
oozie.wf.rerun.failnodes=true

oozieProjectRoot=${nameNode}/oozieProject
appPath=${oozieProjectRoot}/workflowSqoopAction
oozie.wf.application.path=${appPath}

然后我将作业发送到Oozie服务器并开始执行它：

oozie job -oozie http://localhost:11000/oozie -config /home/sergio/git/hadoop_samples/hadoop_examples/src/main/java/org/sanchez/sergio/hadoop_examples/oozie/workflowSqoopAction/job.properties -submit

oozie job -oozie http://localhost:11000/oozie -start 0000001-170415112256550-oozie-serg-W

在Oozie的Web控制台中看到工作失败后不久：

sqoopAction中出现以下错误消息：

JA017: Could not lookup launched hadoop Job ID [job_local245204272_0008] which was associated with  action [0000001-170415112256550-oozie-serg-W@sqoopAction].  Failing this action!

有人可以指导我这个错误吗？

奔跑的恶魔：

jps

2576 
6130 ResourceManager
3267 DataNode
10102 JobHistoryServer
3129 NameNode
24650 Jps
6270 NodeManager
3470 SecondaryNameNode
4190 Bootstrap

Answer 1

您在hadoop中缺少一些配置属性。我也使用hadoop-2.7.3和Oozie-4.3并且在过去5天遇到了同样的问题。

配置上面提到的几个属性，它在我的本地工作：

纱-site.xml中：

<property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
</property>

<property>
    <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>

<property>
    <name>yarn.log-aggregation-enable</name>
    <value>true</value>
</property>

mapred-site.xml中：

<property>
    <name>mapreduce.jobtracker.address</name>
    <value>HOST:PORT</value>
</property>

<property>
    <name>mapreduce.jobtracker.http.address</name>
    <value>HOST:PORT</value>
</property>

<property>
    <name>mapreduce.tasktracker.report.address</name>
    <value>127.0.0.1:0</value>
</property>

<property>
    <name>mapreduce.tasktracker.http.address</name>
    <value>0.0.0.0:50060</value>
</property>

<property>
    <name>mapreduce.job.queuename</name>
    <value>default</value>
</property>

<property>
    <name> mapreduce.framework.name</name>
    <value>yarn</value>
</property>

<property> 
    <name>mapreduce.jobhistory.address</name>
    <value>localhost:10020</value> 
</property>

<property> 
    <name>mapreduce.jobhistory.webapp.address</name>
    <value>localhost:19888</value> 
</property>

根据您的设置，将属性的值替换为实际值。现在重新启动yarn，hadoop和Oozie。

祝你好运。

Oozie Workflow由于错误JA017

1 个答案: