Question

在过去的几天里，我测试了多个版本的Hadoop（1.0.1,1.0.2,1.1.4）。在每种情况下，我都可以使用以下命令行轻松运行WordCount程序：

hadoop  jar  hadoop-examples-1.1.1.jar  wordcount  /input   output

由于上面的命令成功执行，我假设我的Hadoop配置是正确的。但是当我尝试使用Eclipse中完全相同的输入运行程序时，每个单独版本都会收到以下错误消息。 任何人都可以告诉我它不能从Eclipse运行的原因吗？

Dec 12, 2012 2:19:41 PM org.apache.hadoop.util.NativeCodeLoader <clinit>
WARNING: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Dec 12, 2012 2:19:41 PM org.apache.hadoop.mapred.JobClient copyAndConfigureFiles
WARNING: No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
****file:/tmp/wordcount/in
Dec 12, 2012 2:19:42 PM org.apache.hadoop.mapred.JobClient$2 run
INFO: Cleaning up the staging area file:/tmp/hadoop-root/mapred/staging/root-41981592/.staging/job_local_0001
Dec 12, 2012 2:19:42 PM org.apache.hadoop.security.UserGroupInformation doAs
SEVERE: PriviledgedActionException as:root cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: file:/input
Exception in thread "main" org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: file:/input
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:235)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:252)
    at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:962)
    at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:979)
    at org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:174)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:897)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
    at com.igalia.wordcount.WordCount.run(WordCount.java:94)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at com.igalia.wordcount.App.main(App.java:28)

Answer 1

通过配置abject在作业中添加以下两行：

Configuration.addResource(new Path("path-to-your-core-site.xml file"));
Configuration.addResource(new Path("path-to-your-hdfs-site.xml file"));

Answer 2

对于Windows 7上的hadoop-2.2.0，我添加了以下行，它解决了问题（注意：我的Hadoop主页是：C：\ MyWork \ MyProjects \ Hadoop \ hadoop-2.2.0）

配置conf = new Configuration（）;

conf.addResource（新路径（“C：\ MyWork \ MyProjects \ Hadoop \ hadoop-2.2.0 \ etc \ hadoop \ core-site.xml”））;

conf.addResource（新路径（“C：\ MyWork \ MyProjects \ Hadoop \ hadoop-2.2.0 \ etc \ hadoop \ hdfs-site.xml”））;

Hadoop WordCount从命令行运行，但不从Eclipse运行

2 个答案: