尝试运行Pig作业工作流程时出现Oozie错误

时间:2014-05-15 21:32:55

标签: hadoop apache-pig oozie hortonworks-data-platform

我正在尝试使用Oozie 4.0运行简单的Pig作业。我遇到了以下错误。我在位置sharelib的Oozie服务器上创建了/usr/lib/oozie/share/文件夹。 我还在以下位置将同一文件夹推送到HDFS:

/user/hdfs
/user/oozie
/user/mapred

我已经在NameNode,Job Tracker和DataNodes上重新启动了Hadoop服务。我做了这个,Oozie服务停止了,并使用命令将Oozie服务重新启动:

sudo -u oozie /usr/lib/oozie/bin/oozie-start.sh

我仍然收到此错误。

cluster_conf.xml:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<configuration>
<property>
    <name>nameNode</name>
    <value>hdfs://localhost:8020</value>
</property>
<property>
    <name>resourceManager</name>
    <value>localhost:8050</value>
</property>
<property>
    <name>oozie.wf.application.path</name>
    <value>/tmp/workflow.xml</value>
</property>

OozieTestScript.pig

A = load '/tmp/passwd' using PigStorage(':');
B = foreach A generate $0 as id;
dump B;
store B into '/tmp/id5.out';

Pig命令很简单,如下所示。请注意,我将私有IP保留在此处复制的代码之外。

<pig xmlns="uri:oozie:workflow:0.2">
  <job-tracker>172.xx.xx.xx:8050</job-tracker>
  <name-node>hdfs://172.xx.xx.xx:8020</name-node>
  <script>OozieTestScript.pig</script>
</pig>

日志:

2014-05-15 19:47:47,872  WARN PigActionExecutor:542 - USER[mapred] GROUP[-] TOKEN[] APP[Oozie_workflow_test] JOB[0000000-140515194709114-oozie-oozi-W] ACTION[0000000-140515194709114-oozie-oozi-W@pigAction] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.PigMain], main() threw exception, org/apache/pig/Main
2014-05-15 19:47:47,873  WARN PigActionExecutor:542 - USER[mapred] GROUP[-] TOKEN[] APP[Oozie_workflow_test] JOB[0000000-140515194709114-oozie-oozi-W] ACTION[0000000-140515194709114-oozie-oozi-W@pigAction] Launcher exception: org/apache/pig/Main
java.lang.NoClassDefFoundError: org/apache/pig/Main
    at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:324)
    at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:219)
    at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)
    at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)
Caused by: java.lang.ClassNotFoundException: org.apache.pig.Main
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
    ... 17 more

2014-05-15 19:47:47,916 DEBUG ActionCheckXCommand:545 - USER[mapred] GROUP[-] TOKEN[] APP[Oozie_workflow_test] JOB[0000000-140515194709114-oozie-oozi-W] ACTION[0000000-140515194709114-oozie-oozi-W@pigAction] ENDED ActionCheckXCommand for wf actionId=0000000-140515194709114-oozie-oozi-W@pigAction, jobId=0000000-140515194709114-oozie-oozi-W

0 个答案:

没有答案