我正在运行mapreduce作业,但始终无法显示缺少的输入。不幸的是,它没有显示出缺少的地方
from mrjob.job import MRJob
from mrjob.step import MRStep
import re
class flight_combination(MRJob):
def steps(self):
return [MRStep(mapper=self.mapper_1,reducer=self.reducer_1)]
def mapper_1(self,_,value):
group1 = {}
group2 = {}
parts = value.split(",")
destination = parts[0]
origin = parts[1]
count = parts[2]
group1[destination] = {'Origin': origin, 'count': count}
group2[origin] = {'Destination':destination,'count':count}
yield group1
yield group2
def reducer_1(self,key,value):
g1,g2 = data
for key1 in g1:
for key2 in g2:
if g1[key1]['Origin'] == g2[key2]['Destination']:
total = int(g1[key1]['count'])*int(g2[key2]['count'])
yield (key1,key2,total)
if __name__ == '__main__':
flight_combination.run()
以下是错误:
`File "wd.py", line 35, in <module>
flight_combination.run()
…...
File "/usr/lib/python3.6/site-packages/mrjob/job.py", line 536, in run_mapper
for out_key, out_value in mapper(key, value) or ():
ValueError: not enough values to unpack (expected 2, got 1)`
答案 0 :(得分:0)
对象类型run
的{{1}}方法期望2个参数,但提供1个参数。 (默认情况下,Python将flight_combination
作为对象上调用的方法的第一个参数)
要解决此问题-
self
是在父类中定义的,因此请仔细检查其定义并传递另一个参数。run
类来覆盖run
方法,并提供您的逻辑。