我正在尝试使用Oozie 4.0运行简单的Pig作业。我遇到了以下错误。我在位置sharelib
的Oozie服务器上创建了/usr/lib/oozie/share/
文件夹。
我还在以下位置将同一文件夹推送到HDFS:
/user/hdfs
/user/oozie
/user/mapred
我已经在NameNode,Job Tracker和DataNodes上重新启动了Hadoop服务。我做了这个,Oozie服务停止了,并使用命令将Oozie服务重新启动:
sudo -u oozie /usr/lib/oozie/bin/oozie-start.sh
我仍然收到此错误。
cluster_conf.xml:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<configuration>
<property>
<name>nameNode</name>
<value>hdfs://localhost:8020</value>
</property>
<property>
<name>resourceManager</name>
<value>localhost:8050</value>
</property>
<property>
<name>oozie.wf.application.path</name>
<value>/tmp/workflow.xml</value>
</property>
OozieTestScript.pig
A = load '/tmp/passwd' using PigStorage(':');
B = foreach A generate $0 as id;
dump B;
store B into '/tmp/id5.out';
Pig命令很简单,如下所示。请注意,我将私有IP保留在此处复制的代码之外。
<pig xmlns="uri:oozie:workflow:0.2">
<job-tracker>172.xx.xx.xx:8050</job-tracker>
<name-node>hdfs://172.xx.xx.xx:8020</name-node>
<script>OozieTestScript.pig</script>
</pig>
日志:
2014-05-15 19:47:47,872 WARN PigActionExecutor:542 - USER[mapred] GROUP[-] TOKEN[] APP[Oozie_workflow_test] JOB[0000000-140515194709114-oozie-oozi-W] ACTION[0000000-140515194709114-oozie-oozi-W@pigAction] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.PigMain], main() threw exception, org/apache/pig/Main
2014-05-15 19:47:47,873 WARN PigActionExecutor:542 - USER[mapred] GROUP[-] TOKEN[] APP[Oozie_workflow_test] JOB[0000000-140515194709114-oozie-oozi-W] ACTION[0000000-140515194709114-oozie-oozi-W@pigAction] Launcher exception: org/apache/pig/Main
java.lang.NoClassDefFoundError: org/apache/pig/Main
at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:324)
at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:219)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)
at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)
Caused by: java.lang.ClassNotFoundException: org.apache.pig.Main
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 17 more
2014-05-15 19:47:47,916 DEBUG ActionCheckXCommand:545 - USER[mapred] GROUP[-] TOKEN[] APP[Oozie_workflow_test] JOB[0000000-140515194709114-oozie-oozi-W] ACTION[0000000-140515194709114-oozie-oozi-W@pigAction] ENDED ActionCheckXCommand for wf actionId=0000000-140515194709114-oozie-oozi-W@pigAction, jobId=0000000-140515194709114-oozie-oozi-W