当我试图运行oozie Job时,它会抛出以下错误 Launcher ERROR,原因:主类[org.apache.oozie.action.hadoop.ShellMain],退出代码[1]
我的shell脚本在独立运行时运行良好。
我是oozie的新手,非常感谢任何帮助
<workflow-app name="DQ-wf" xmlns="uri:oozie:workflow:0.4">
<start to = 'validate-tables' />
<action name="validate-tables">
<shell xmlns="uri:oozie:shell-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
<exec>test.sh</exec>
<file>test.sh</file>
<capture-output/>
</shell>
<ok to="decide"/>
<error to="fail"/>
</action>
<decision name="decide">
<switch>
<case to="shell-file">
${wf:actionData("shell-file")["isMatch"]}
</case>
<default to="end"/>
</switch>
</decision>
<action name="shell-file">
<shell xmlns="uri:oozie:shell-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
<exec>copy-output.sh</exec>
<file>copy-output.sh</file>
</shell>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Workflow failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name='end' />
</workflow-app>
Job.Properties
===============
nameNode=hdfs://dev-devwork:8020
jobTracker=isperf.xxx.com:8050
hdfs=hdfs://dev-devwork
queueName=default
workflowRoot=/etc/hive/conf
oozie.use.system.libpath=true
oozieProjectRoot=${nameNode}/user/gdsds/oozie/ing
appPath=${oozieProjectRoot}
oozie.wf.application.path=${appPath}
test.sh
========
DATE=`date +%Y_%m_%d_%H_%M_%S`;
. ./env_settings.ini
destdir=/home/gdsds/oozie/$DATE.txt
Q1=$( hive -S -e "select count(1) from gpc.TABLE_A;")
echo $Q1;
Q1T=$( hive -S -e "select count(1) from gpc.TABLE_B;")
if [ "$Q1" -eq "$Q1T" ]
then
echo "$Q1 TABLE_A VS TABLE_B MATCH" >> "$destdir";
else
echo "$Q1 TABLE_A VS TABLE_B UNMATCH" >> "$destdir";
echo "isMatch=false";
exit 0
fi
Q2=$( hive -S -e "select count(1) from gpc.TABLE_X;")
echo $Q2
Q2T=$( hive -S -e "select count(1) from gpc.TABLE_Y")
if [ "$Q2" -eq "$Q2T" ]
then
echo "$Q2 TABLE_X VS TABLE_Y MATCH" >> "$destdir";
echo "isMatch=true";
else
echo "$Q2 TABLE_X VS TABLE_Y UNMATCH" >> "$destdir";
echo "isMatch=false";
exit 0
fi
2016-05-04 23:04:47,771 INFO ActionStartXCommand:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W@:start:] Start action [0186090-160418004347476-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-05-04 23:04:47,774 INFO ActionStartXCommand:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W@:start:] [***0186090-160418004347476-oozie-oozi-W@:start:***]Action status=DONE
2016-05-04 23:04:47,774 INFO ActionStartXCommand:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W@:start:] [***0186090-160418004347476-oozie-oozi-W@:start:***]Action updated in DB!
2016-05-04 23:04:47,799 INFO WorkflowNotificationXCommand:520 - SERVER[isperf.xxx.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W@:start:] No Notification URL is defined. Therefore nothing to notify for job 0186090-160418004347476-oozie-oozi-W@:start:
2016-05-04 23:04:47,801 INFO WorkflowNotificationXCommand:520 - SERVER[isperf.xxx.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0186090-160418004347476-oozie-oozi-W
2016-05-04 23:04:47,817 INFO ActionStartXCommand:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W@validate-tables] Start action [0186090-160418004347476-oozie-oozi-W@validate-tables] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-05-04 23:04:49,257 INFO ShellActionExecutor:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W@validate-tables] checking action, hadoop job ID [job_1461261058027_467475] status [RUNNING]
2016-05-04 23:04:49,258 INFO ActionStartXCommand:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W@validate-tables] [***0186090-160418004347476-oozie-oozi-W@validate-tables***]Action status=RUNNING
2016-05-04 23:04:49,258 INFO ActionStartXCommand:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W@validate-tables] [***0186090-160418004347476-oozie-oozi-W@validate-tables***]Action updated in DB!
2016-05-04 23:04:49,262 INFO WorkflowNotificationXCommand:520 - SERVER[isperf.xxx.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W@validate-tables] No Notification URL is defined. Therefore nothing to notify for job 0186090-160418004347476-oozie-oozi-W@validate-tables
2016-05-04 23:05:16,380 INFO CallbackServlet:520 - SERVER[isperf.xxx.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W@validate-tables] callback for action [0186090-160418004347476-oozie-oozi-W@validate-tables]
2016-05-04 23:05:16,550 INFO ShellActionExecutor:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W@validate-tables] action completed, external ID [job_1461261058027_467475]
2016-05-04 23:05:16,554 WARN ShellActionExecutor:523 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W@validate-tables] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.ShellMain], exit code [1]
2016-05-04 23:05:16,600 INFO ActionEndXCommand:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W@validate-tables] ERROR is considered as FAILED for SLA
2016-05-04 23:05:16,627 INFO ActionStartXCommand:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W@fail] Start action [0186090-160418004347476-oozie-oozi-W@fail] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-05-04 23:05:16,630 INFO ActionStartXCommand:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W@fail] [***0186090-160418004347476-oozie-oozi-W@fail***]Action status=DONE
2016-05-04 23:05:16,631 INFO ActionStartXCommand:520 - SERVER[isperf.xxx.com] USER[gdsds] GROUP[-] TOKEN[] APP[DQ-wf] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W@fail] [***0186090-160418004347476-oozie-oozi-W@fail***]Action updated in DB!
2016-05-04 23:05:16,678 INFO WorkflowNotificationXCommand:520 - SERVER[isperf.xxx.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W@fail] No Notification URL is defined. Therefore nothing to notify for job 0186090-160418004347476-oozie-oozi-W@fail
2016-05-04 23:05:16,678 INFO WorkflowNotificationXCommand:520 - SERVER[isperf.xxx.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0186090-160418004347476-oozie-oozi-W
2016-05-04 23:05:16,678 INFO WorkflowNotificationXCommand:520 - SERVER[isperf.xxx.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0186090-160418004347476-oozie-oozi-W] ACTION[0186090-160418004347476-oozie-oozi-W@validate-tables] No Notification URL is defined. Therefore nothing to notify for job 0186090-160418004347476-oozie-oozi-W@validate-tables