我一直在SSH上使用Hadoop MapReduce命令,如下所示:
hadoop jar /usr/lib/hadoop-0.20-mapreduce/contrib/streaming/hadoop-streaming-2.0.0-mr1-cdh4.0.1.jar -file /nfs_home/appers/user1/mapper.py -file /nfs_home/appers/user1/reducer.py -mapper '/usr/lib/python_2.7.3/bin/python mapper.py' -reducer '/usr/lib/python_2.7.3/bin/python reducer.py' -input /ccexp/data/test_xml/0901282-510179094535002-oozie-oozi-W/extractOut//.xml -output /user/ccexptest/output/user1/MRoutput
但是,我正在扩展Mapper.py脚本的功能,并希望在启动MapReduce作业时将字符串参数传递给Mapper。如何编辑上面的MR命令,以便将字符串作为参数包含在Mapper中?
答案 0 :(得分:1)
根据您的情况,您可以将任意命令行输入传递到-mapper
和-reducer
-mapper '/usr/lib/python_2.7.3/bin/python mapper.py myargs'