hadoop编译 - 在dfs文件中

Question

我想用以下命令编译hadoop示例，但是会发生错误：

$ mkdir wordcount_classes
$ javac -classpath ${HADOOP_HOME}/hadoop-${HADOOP_VERSION}-core.jar -d wordcount_classes WordCount.java
$ jar -cvf /usr/joe/wordcount.jar -C wordcount_classes/ .

Assuming that:

    /usr/joe/wordcount/input - input directory in HDFS
    /usr/joe/wordcount/output - output directory in HDFS

Sample text-files as input:

$ bin/hadoop dfs -ls /usr/joe/wordcount/input/
/usr/joe/wordcount/input/file01
/usr/joe/wordcount/input/file02

$ bin/hadoop dfs -cat /usr/joe/wordcount/input/file01
Hello World Bye World

$ bin/hadoop dfs -cat /usr/joe/wordcount/input/file02
Hello Hadoop Goodbye Hadoop 

Answer 1

/usr/joe是本地的，正如您在第一行执行ls命令时所看到的那样。第二个命令需要HDFS位置的输入和输出，但HDFS上不存在/usr/joe。您需要将数据移动到HDFS上，然后执行命令。例如：

#This creates a folder "wordcount/input" in your HDFS home directory
hdfs dfs -mkdir -p wordcount/input
hdfs dfs -put /usr/joe/wordcount/input/* wordcount/input