我正在尝试运行Ruby Hadoop流程序，该程序在“Ruby的权威指南”中给出。
这是我使用的命令：
hadoop jar /usr/lib/hadoop-0.20/contrib/streaming/hadoop-streaming-0.20.2+737.jar
-input input/temperature -output output -mapper /home/cloudera/projects/max_temp/map.rb
-reducer /home/cloudera/projects/max_temp/reduce.rb
文件路径正确。运行命令后，我收到如下错误：
packageJobJar: [/var/lib/hadoop-0.20/cache/cloudera/hadoop-unjar7505856816337735719/] [] /tmp/streamjob5440613434295562761.jar tmpDir=null
11/03/14 22:58:56 INFO mapred.FileInputFormat: Total input paths to process : 1
11/03/14 22:58:56 INFO streaming.StreamJob: getLocalDirs(): [/var/lib/hadoop-0.20/cache/cloudera/mapred/local]
11/03/14 22:58:56 INFO streaming.StreamJob: Running job: job_201103140235_0015
11/03/14 22:58:56 INFO streaming.StreamJob: To kill this job, run:
11/03/14 22:58:56 INFO streaming.StreamJob: /usr/lib/hadoop-0.20/bin/hadoop job -Dmapred.job.tracker=localhost:8021 -kill job_201103140235_0015
11/03/14 22:58:56 INFO streaming.StreamJob: Tracking URL: http://localhost:50030/jobdetails.jsp...
11/03/14 22:58:57 INFO streaming.StreamJob: map 0% reduce 0%
11/03/14 22:59:54 INFO streaming.StreamJob: map 100% reduce 100%
11/03/14 22:59:54 INFO streaming.StreamJob: To kill this job, run:
11/03/14 22:59:54 INFO streaming.StreamJob: /usr/lib/hadoop-0.20/bin/hadoop job -Dmapred.job.tracker=localhost:8021 -kill job_201103140235_0015
11/03/14 22:59:54 INFO streaming.StreamJob: Tracking URL: http://localhost:50030/jobdetails.jsp...
11/03/14 22:59:54 ERROR streaming.StreamJob: Job not Successful!
11/03/14 22:59:54 INFO streaming.StreamJob: killJob...
Streaming Command Failed!
我查看了地图日志文件，这是错误：
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:379)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:317)
at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
at org.apache.hadoop.mapred.Child.main(Child.java:211)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 9 more
Caused by: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 14 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 17 more
Caused by: java.lang.RuntimeException: configuration exception
at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:230)
at org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66)
... 22 more
Caused by: java.io.IOException: Cannot run program "/home/cloudera/projects/max_temp/map.rb": java.io.IOException: error=13, Permission denied
at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:214)
... 23 more
Caused by: java.io.IOException: java.io.IOException: error=13, Permission denied
at java.lang.UNIXProcess.(UNIXProcess.java:148)
at java.lang.ProcessImpl.start(ProcessImpl.java:65)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
... 24 more
答案 0 :(得分：4)
感谢Lev Khomich的回复。我通过在命令中提到file参数来修复此问题：
hadoop jar /usr/lib/hadoop-0.20/contrib/streaming/hadoop-streaming-0.20.2+737.jar \
-input input/temperature -output output -mapper /home/cloudera/projects/max_temp/map.rb \
-reducer /home/cloudera/projects/max_temp/reduce.rb \
-file /home/cloudera/projects/words/map.rb \
-file /home/cloudera/projects/words/reduce.rb
答案 1 :(得分：0)
我和你有同样的错误，并通过将'map.rb'和'reduce.rb'文件复制到/ tmp /修复问题，然后将parms作为-mapper /tmp/map.rb -reducer /tmp/reduce.rb运行