Hadoop 1.2.1中的BIgData中的流命令失败

时间:2018-11-10 14:52:25

标签: python bigdata hadoop-streaming

我正在处理bigdata hadoop,并且获取流命令失败,所以请      帮我。

还附加地图并减少代码

 mapper code-

 #!/usr/bin/python2
 import sys

 k=[]
 for i in sys.stdin:
 k.extend(i.split())
 print (k)
 reducer code-

enter code here #!/usr/bin/python2

import sys
fcounter=0
mcounter=0
M=[]
for k in sys.stdin:
     M=k
print(M+"\n")
   for n in M:
   if 'F'==n:
            fcounter+=1
   if'M'==n:
            mcounter+=1

   tot=mcounter+fcounter
print ("Total no. of Candidates:-"+str(tot)/2)      
print ('F:',int(fcounter/2)+1)
print ('M:',int(mcounter/2)+1)

//this is the streaming command
 hadoop jar /usr/share/hadoop/contrib/streaming/hadoop-streaming-1.2.1.jar - 
 mapper mapper.py -file mapper.py -reducer reducer.py -file reducer.py - 
 input /Aadhar_detail.txt -output / aadharop.txt 

还附加地图并减少代码

0 个答案:

没有答案