Question

我有以下数据框

userID  movieID rating  timestamp
1   1   9   12
1   2   10  13

我调用了这个dataframe mapper1.txt并将其存储在与此python文件相同的目录中：

from mrjob.job import MRJob

class MRRatingCounter(MRJob):
    def mapper(self, key, line):
        (userID, movieID, rating, timestamp) = line.split('\t')
        yield rating, 1

    def reducer(self, rating, occurences):
        yield rating, sum(occurences)

if __name__ == '__main__':
    MRRatingCounter.run()

不，我想运行以下代码来运行此功能

!python Rating-Counter.py mapreduce\mapper1.txt

然而，这会引发以下错误：

!python Rating-Counter.py mapreduce\mapper1.txt
no configs found; falling back on auto-configuration
no configs found; falling back on auto-configuration
Traceback (most recent call last):
  File "Rating-Counter.py", line 12, in <module>
    MRRatingCounter.run()
  File "C:\Users\Marc\AppData\Local\Enthought\Canopy\User\lib\site-packages\mrjob\job.py", line 461, in run
    mr_job.execute()
  File "C:\Users\Marc\AppData\Local\Enthought\Canopy\User\lib\site-packages\mrjob\job.py", line 479, in execute
    super(MRJob, self).execute()
  File "C:\Users\Marc\AppData\Local\Enthought\Canopy\User\lib\site-packages\mrjob\launch.py", line 153, in execute
    self.run_job()
  File "C:\Users\Marc\AppData\Local\Enthought\Canopy\User\lib\site-packages\mrjob\launch.py", line 216, in run_job
    runner.run()
  File "C:\Users\Marc\AppData\Local\Enthought\Canopy\User\lib\site-packages\mrjob\runner.py", line 470, in run
    self._run()
  File "C:\Users\Marc\AppData\Local\Enthought\Canopy\User\lib\site-packages\mrjob\sim.py", line 164, in _run
    _error_on_bad_paths(self.fs, self._input_paths)
  File "C:\Users\Marc\AppData\Local\Enthought\Canopy\User\lib\site-packages\mrjob\sim.py", line 549, in _error_on_bad_paths
    "None found in %s" % paths)
ValueError: At least one valid path is required. None found in ['mapreduce\\mapper1.txt']

!python Rating-Counter.py mapreduce\mapper1.txt

但我不明白什么是错的。有人可以向我解释为什么这段代码不起作用吗？

Answer 1

输入文件内容应以制表符分隔..

| userID | movieID | rating | timestamp |
|--------|---------|--------|-----------|
| 1      | 1       | 9      | 12        |
| 1      | 2       | 10     | 13        |

要运行的命令是

!python Rating-Counter.py mapper1.txt

确保py文件和txt文件位于相同的文件夹/当前目录中

我的输出：

C:\>python mrjobfirst.py
No configs found; falling
Creating temp directory c
0170424.081057.565000
Running step 1 of 1...
Streaming final output fr
mm.20170424.081057.565000
"10"    1
"9"     1
"rating"        1
Removing temp directory c
0170424.081057.565000...

运行mapreduce函数时出错

1 个答案: