Eclipse环境中的Hadoop

时间：2012-07-13 09:33:31

标签： java linux eclipse ssh hadoop

运行http://developer.yahoo.com/hadoop/tutorial/module3.html#running提供的wordcount示例 我在yahoo提供的VM上运行hadoop，并通过ssh客户端连接到VM。

我收到了这些错误：

12/07/13 00:10:03 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
12/07/13 00:10:03 INFO mapred.FileInputFormat: Total input paths to process : 2
12/07/13 00:10:03 INFO mapred.FileInputFormat: Total input paths to process : 2
12/07/13 00:10:04 INFO mapred.JobClient: Running job: job_201207121015_0021
12/07/13 00:10:05 INFO mapred.JobClient:  map 0% reduce 0%
12/07/13 00:10:10 INFO mapred.JobClient: Task Id : attempt_201207121015_0021_m_000000_0, Status : FAILED
java.lang.UnsupportedClassVersionError: WordCountReducer : Unsupported major.minor version 51.0
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(Unknown Source)
    at java.security.SecureClassLoader.defineClass(Unknown Source)
    at java.net.URLClassLoader.defineClass(Unknown Source)
    at java.net.URLClassLoader.access$000(Unknown Source)
    at java.net.URLClassLoader$1.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at java.lang.ClassLoader.loadClassInternal(Unknown Source)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Unknown Source)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:628)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:646)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:672)
    at org.apache.hadoop.mapred.JobConf.getCombinerClass(JobConf.java:780)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:403)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:193)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2209)

12/07/13 00:10:10 WARN mapred.JobClient: Error reading task outputhadoop-desk.localdomain
12/07/13 00:10:10 WARN mapred.JobClient: Error reading task outputhadoop-desk.localdomain
12/07/13 00:10:10 INFO mapred.JobClient: Task Id : attempt_201207121015_0021_m_000001_0, Status : FAILED
java.lang.UnsupportedClassVersionError: WordCountReducer : Unsupported major.minor version 51.0
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(Unknown Source)
    at java.security.SecureClassLoader.defineClass(Unknown Source)
    at java.net.URLClassLoader.defineClass(Unknown Source)
    at java.net.URLClassLoader.access$000(Unknown Source)
    at java.net.URLClassLoader$1.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at java.lang.ClassLoader.loadClassInternal(Unknown Source)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Unknown Source)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:628)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:646)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:672)
    at org.apache.hadoop.mapred.JobConf.getCombinerClass(JobConf.java:780)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:403)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:193)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2209)

12/07/13 00:10:10 WARN mapred.JobClient: Error reading task outputhadoop-desk.localdomain
12/07/13 00:10:10 WARN mapred.JobClient: Error reading task outputhadoop-desk.localdomain
12/07/13 00:10:14 INFO mapred.JobClient: Task Id : attempt_201207121015_0021_m_000000_1, Status : FAILED
java.lang.UnsupportedClassVersionError: WordCountReducer : Unsupported major.minor version 51.0
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(Unknown Source)
    at java.security.SecureClassLoader.defineClass(Unknown Source)
    at java.net.URLClassLoader.defineClass(Unknown Source)
    at java.net.URLClassLoader.access$000(Unknown Source)
    at java.net.URLClassLoader$1.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at java.lang.ClassLoader.loadClassInternal(Unknown Source)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Unknown Source)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:628)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:646)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:672)
    at org.apache.hadoop.mapred.JobConf.getCombinerClass(JobConf.java:780)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:403)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:193)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2209)

12/07/13 00:10:14 WARN mapred.JobClient: Error reading task outputhadoop-desk.localdomain
12/07/13 00:10:14 WARN mapred.JobClient: Error reading task outputhadoop-desk.localdomain
12/07/13 00:10:15 INFO mapred.JobClient: Task Id : attempt_201207121015_0021_m_000001_1, Status : FAILED
java.lang.UnsupportedClassVersionError: WordCountReducer : Unsupported major.minor version 51.0
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(Unknown Source)
    at java.security.SecureClassLoader.defineClass(Unknown Source)
    at java.net.URLClassLoader.defineClass(Unknown Source)
    at java.net.URLClassLoader.access$000(Unknown Source)
    at java.net.URLClassLoader$1.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at java.lang.ClassLoader.loadClassInternal(Unknown Source)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Unknown Source)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:628)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:646)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:672)
    at org.apache.hadoop.mapred.JobConf.getCombinerClass(JobConf.java:780)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:403)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:193)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2209)

12/07/13 00:10:15 WARN mapred.JobClient: Error reading task outputhadoop-desk.localdomain
12/07/13 00:10:15 WARN mapred.JobClient: Error reading task outputhadoop-desk.localdomain
12/07/13 00:10:19 INFO mapred.JobClient: Task Id : attempt_201207121015_0021_m_000000_2, Status : FAILED
java.lang.UnsupportedClassVersionError: WordCountReducer : Unsupported major.minor version 51.0
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(Unknown Source)
    at java.security.SecureClassLoader.defineClass(Unknown Source)
    at java.net.URLClassLoader.defineClass(Unknown Source)
    at java.net.URLClassLoader.access$000(Unknown Source)
    at java.net.URLClassLoader$1.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at java.lang.ClassLoader.loadClassInternal(Unknown Source)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Unknown Source)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:628)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:646)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:672)
    at org.apache.hadoop.mapred.JobConf.getCombinerClass(JobConf.java:780)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:403)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:193)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2209)

12/07/13 00:10:19 WARN mapred.JobClient: Error reading task outputhadoop-desk.localdomain
12/07/13 00:10:19 WARN mapred.JobClient: Error reading task outputhadoop-desk.localdomain
12/07/13 00:10:19 INFO mapred.JobClient: Task Id : attempt_201207121015_0021_m_000001_2, Status : FAILED
java.lang.UnsupportedClassVersionError: WordCountReducer : Unsupported major.minor version 51.0
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(Unknown Source)
    at java.security.SecureClassLoader.defineClass(Unknown Source)
    at java.net.URLClassLoader.defineClass(Unknown Source)
    at java.net.URLClassLoader.access$000(Unknown Source)
    at java.net.URLClassLoader$1.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at java.lang.ClassLoader.loadClassInternal(Unknown Source)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Unknown Source)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:628)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:646)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:672)
    at org.apache.hadoop.mapred.JobConf.getCombinerClass(JobConf.java:780)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:403)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:193)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2209)

12/07/13 00:10:19 WARN mapred.JobClient: Error reading task outputhadoop-desk.localdomain
12/07/13 00:10:19 WARN mapred.JobClient: Error reading task outputhadoop-desk.localdomain
java.io.IOException: Job failed!
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1113)
    at WordCount.main(WordCount.java:33)

1 个答案:

答案 0 :(得分：1)

您正在使用Java7编译代码，而Hadoop正在Java 6上运行。您可以让Hadoop在Java7上运行，也可以为Java 6编译代码。

相关问题
最新问题