Question

我收到以下错误：当我尝试执行用Python编写的mapreduce作业时...我无法找到流* .jar ...请建议如何解决此问题.. 能否请你指导我写下以下命令的bash文件..

hduser@hadoop-PC:~/hadoop/contrib$ hadoop jar contrib/streaming/hadoop-*streaming*.jar -file /home/hduser/mapper.py    -mapper /home/hduser/mapper.py -file /home/hduser/reducer.py   -reducer /home/hduser/reducer.py -input /user/hduser/gutenberg/* -output /user/hduser/gutenberg-output
Warning: $HADOOP_HOME is deprecated.

Exception in thread "main" java.io.IOException: Error opening job jar: contrib/streaming/hadoop-*streaming*.jar
    at org.apache.hadoop.util.RunJar.main(RunJar.java:90)
Caused by: java.io.FileNotFoundException: contrib/streaming/hadoop-*streaming*.jar (No such file or directory)
    at java.util.zip.ZipFile.open(Native Method)
    at java.util.zip.ZipFile.<init>(ZipFile.java:215)
    at java.util.zip.ZipFile.<init>(ZipFile.java:145)
    at java.util.jar.JarFile.<init>(JarFile.java:153)
    at java.util.jar.JarFile.<init>(JarFile.java:90)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:88)

Answer 1

嗯，愚蠢的问题，但你确认A）文件存在和B）你有权访问它吗？另外，尝试访问./contrib/streaming/hadoop-streaming.jar之类的文件，甚至是访问它的完全限定路径。

我刚刚注意到您从~/hadoop/contrib运行了代码。您需要从路径中取消前导contrib

Answer 2

嗯，我想，不是hadoop-streaming.jar，它应该是hadoop-streaming-1.1.2.jar，所以试试这个：

hduser@hadoop-PC:~/hadoop$./bin/hadoop jar contrib/streaming/hadoop-streaming-1.1.2.jar** -file /home/hduser/mapper.py    -mapper /home/hduser/mapper.py -file /home/hduser/reducer.py   -reducer /home/hduser/reducer.py -input /user/hduser/gutenberg/* -output /user/hduser/gutenberg-output

hadoop 1.1.2 - 找不到流媒体jar

2 个答案: