Hadoop:ClassNotFoundException $ Reduce

时间:2013-12-07 17:00:21

标签: java hadoop

我有一个由Jetty服务器的handle方法调用的字数示例类。工作提交时:

JobClient.runJob(conf);

可悲的是,JobTracker什么都没看到,工作没有提交,但它完成并创建了输出。我发现,我必须注入我的配置:

conf.addResource(new Path("/usr/local/hadoop/conf/mapred-site.xml"));

添加此行后,JobTracker获取作业,但失败并出现以下异常:

INFO: Task Id : attempt_201312071601_0005_m_000000_1, Status : FAILED
java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: WC2$Reduce
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:889)
    at org.apache.hadoop.mapred.JobConf.getCombinerClass(JobConf.java:1049)
    at org.apache.hadoop.mapred.Task$CombinerRunner.create(Task.java:1385)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:986)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:422)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: WC2$Reduce
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:881)
    ... 10 more
Caused by: java.lang.ClassNotFoundException: WC2$Reduce
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:270)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855)
    ... 11 more

我的组合器类与reducer类相同。我没有修改就下载了Hadoop WordCount示例。只提到addResource

0 个答案:

没有答案