Question

刚刚安装了Pig 0.13，我试图将它与Hadoop 1.1.2一起使用。（Pig文档说Pig 0.13与Hadoop 1.1.2兼容）。根据Pig安装说明，我设置了$ PIG_CLASSPATH 指向/ etc / hadoop，其中定义了core-site.xml，hdfs-site.xml和mapred-site.xml。 Hadoop集群功能齐全，适用于非Pig作业。基于下面的错误描述，我理解Pig找不到它正在寻找的JobContextImpl类。

基于Hadoop 1.1.2 API文档，我不相信＆＃34;任务＆＃34;是＆＃34; mapreduce＆＃34;的一个子包。包。我尝试将hadoop-core-1.1.2.jar直接添加到$ PIG_CLASSPATH 那不起作用。（在查看hadoop-core-1.1.2.jar和Hadoop 1.1.2 API文档的内容之后，我不相信在Pig试图从中加载它的包中定义了JobContextImpl）。如何让Pig 0.13与Hadoop 1.1.2一起使用？

=======错误如下：=======

14/08/03 14:01:05 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
14/08/03 14:01:05 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
14/08/03 14:01:05 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the ExecType
2014-08-03 14:01:05,959 [main] INFO  org.apache.pig.Main - Apache Pig version 0.13.0 (r1606446) compiled Jun 29 2014, 02:27:58
2014-08-03 14:01:05,959 [main] INFO  org.apache.pig.Main - Logging error messages to: /home/hadoop/pig-0.13.0/bin/pig_1407088865958.log
2014-08-03 14:01:06,112 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://master.localdomain:8020/
2014-08-03 14:01:06,388 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: master.localdomain:8021
2014-08-03 14:01:06,440 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2998: Unhandled internal error. org/apache/hadoop/mapreduce/task/JobContextImpl
Details at logfile: /home/hadoop/pig-0.13.0/bin/pig_1407088865958.log
Exception in thread "main" java.lang.NoClassDefFoundError: Could not initialize class org.apache.pig.tools.pigstats.PigStatsUtil
    at org.apache.pig.Main.run(Main.java:643)
    at org.apache.pig.Main.main(Main.java:156)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

=== pig_1407088865958.log ===

的内容

猪堆痕迹

ERROR 2998: Unhandled internal error. org/apache/hadoop/mapreduce/task/JobContextImpl
java.lang.NoClassDefFoundError: org/apache/hadoop/mapreduce/task/JobContextImpl
    at org.apache.pig.tools.pigstats.PigStatsUtil.<clinit>(PigStatsUtil.java:68)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:79)
    at org.apache.pig.Main.run(Main.java:510)
    at org.apache.pig.Main.main(Main.java:156)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.task.JobContextImpl
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 9 more

Answer 1

虽然目前尚不清楚这对每个人的效果如何，但似乎提问者提到了他是如何解决问题的：

在我寻求帮助时，我看到帖子说明它需要使用指示版本的参数重新编译。参数我看到的价值是23,24。我不知道该参数如何映射到我正在使用的hadoop版本1.1.2。我砍了垃圾桶/猪脚本指向hadoop-core-1.1.2.jar。脚本需要设置HADOOP_HOME（不推荐使用）。

Pig 0.13 ERROR 2998：未处理的内部错误。组织/阿帕奇/的Hadoop / MapReduce的/任务/ JobContextImpl

猪堆痕迹

1 个答案: