运行jar时ClassNotFoundException虽然它包含所有类

Question

我使用jar为eclipse个作业创建了mapreduce。如果你提取jar，你可以看到那里的所有课程。当您run jar hadoop`命令hadoop``using时，它会显示如下所示的错误。

它无法仅识别一个类，即Test_project$TwoDArrayWritables。 Test_project类是main，TwoDArrayWritables中的class是Test_project。 TwoDArrayWritables继承TwoDArrayWritable中的hadoop内置类。

Jar extract图片：

错误：

16/04/05 15:48:28 INFO mapred.JobClient: Task Id : attempt_201604051120_0002_m_000000_1, Status : FAILED
java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: mapreduce.Test_project$TwoDArrayWritables
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:889)
    at org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:747)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:966)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:422)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: mapreduce.Test_project$TwoDArrayWritables
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:881)
    ... 9 more
Caused by: java.lang.ClassNotFoundException: mapreduce.Test_project$TwoDArrayWritables
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855)
    ... 10 more

16/04/05 15:48:34 INFO mapred.JobClient: Task Id : attempt_201604051120_0002_m_000000_2, Status : FAILED
java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: mapreduce.Test_project$TwoDArrayWritables
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:889)
    at org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:747)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:966)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:422)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: mapreduce.Test_project$TwoDArrayWritables
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:881)
    ... 9 more
Caused by: java.lang.ClassNotFoundException: mapreduce.Test_project$TwoDArrayWritables
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855)
    ... 10 more

用Google搜索了许多解决方案，但没有任何效果。请帮忙！

Answer 1

试试这个：bin/hadoop jar /path/to/jarfile/newproj.jar Test_project.TwoDArrayWritables /user/hduser/input /user/hduser/output1

Answer 2

苦苦挣扎了几个星期，在我的脑海里点了点什么， 我在我的工作中使用了两个reducer，因此为每个都定义了两个Jobconf：

我之前的（错误代码）

JobConf conf = new JobConf(getConf(), Test_project.class); JobConf conf2 = new JobConf(getConf());

我认为配置已经定义，所以没有在 conf2 中提及 Test_project.class

我的礼物（正确的代码）

JobConf conf = new JobConf(getConf(), Test_project.class); JobConf conf2 = new JobConf(getConf(), Test_project.class);

错误被抛出，因为在执行时它正在搜索Test_project $ TwoDArrayWritables，因为它没有Test_project.class类，它找不到 TwoDArrayWritables

现在工作正常。