Question

如果我正确理解MRJob，您可以使用MRJob模拟hadoop的多进程运行，并运行

python mrfile.py -r local input.txt

我正在运行Windows（现在别无选择），当我发出上述命令时，我得到了一堆mambo jumbo，最后它告诉我：

WindowsError: [Error 2] The system cannot find the file specified

这是完整的错误。有人可以帮忙吗？

C:\Users\someuser\Documents\Python_projects\something>python MRJob_parser.py -r loc
al test2.txt
no configs found; falling back on auto-configuration
no configs found; falling back on auto-configuration
creating tmp directory c:\users\someuser\appdata\local\temp\MRJob_parser.someuser.
20150701.211822.496000
writing wrapper script to c:\users\someuser\appdata\local\temp\MRJob_parser.bw401
45.20150701.211822.496000\setup-wrapper.sh
writing to c:\users\someuser\appdata\local\temp\MRJob_parser.someuser.20150701.211
822.496000\step-0-mapper_part-00000
> sh -ex setup-wrapper.sh 'c:\Users\someuser\Documents\python_venv\something_Project\
Scripts\python.exe' MRJob_parser.py --step-num=0 --mapper 'c:\users\someuser\appd
ata\local\temp\MRJob_parser.someuser.20150701.211822.496000\input_part-00000' > c
:\users\someuser\appdata\local\temp\MRJob_parser.someuser.20150701.211822.496000\s
tep-0-mapper_part-00000
Traceback (most recent call last):
  File "MRJob_parser.py", line 18, in <module>
    Extractor.run()
  File "c:\Users\someuser\Documents\python_venv\something_Project\lib\site-packages\m
rjob\job.py", line 461, in run
    mr_job.execute()
  File "c:\Users\someuser\Documents\python_venv\something_Project\lib\site-packages\m
rjob\job.py", line 479, in execute
    super(MRJob, self).execute()
  File "c:\Users\someuser\Documents\python_venv\something_Project\lib\site-packages\m
rjob\launch.py", line 151, in execute
    self.run_job()
  File "c:\Users\someuser\Documents\python_venv\something_Project\lib\site-packages\m
rjob\launch.py", line 214, in run_job
    runner.run()
  File "c:\Users\someuser\Documents\python_venv\something_Project\lib\site-packages\m
rjob\runner.py", line 464, in run
    self._run()
  File "c:\Users\someuser\Documents\python_venv\something_Project\lib\site-packages\m
rjob\sim.py", line 173, in _run
    self._invoke_step(step_num, 'mapper')
  File "c:\Users\someuser\Documents\python_venv\something_Project\lib\site-packages\m
rjob\sim.py", line 260, in _invoke_step
    working_dir, env)
  File "c:\Users\someuser\Documents\python_venv\something_Project\lib\site-packages\m
rjob\local.py", line 147, in _run_step
    procs_args, output_path, working_dir, env)
  File "c:\Users\someuser\Documents\python_venv\something_Project\lib\site-packages\m
rjob\local.py", line 250, in _invoke_processes
    cwd=working_dir, env=env)
  File "c:\Users\someuser\Documents\python_venv\something_Project\lib\site-packages\m
rjob\local.py", line 73, in _chain_procs
    proc = Popen(args, **proc_kwargs)
  File "C:\Python27\Lib\subprocess.py", line 710, in __init__
    errread, errwrite)
  File "C:\Python27\Lib\subprocess.py", line 958, in _execute_child
    startupinfo)
WindowsError: [Error 2] The system cannot find the file specified

C:\Users\someuser\Documents\Python_projects\something>

Answer 1

来自https://github.com/Yelp/mrjob：

# locally
python mrjob/examples/mr_word_freq_count.py README.rst > counts

我的猜测是你不必传入本地跑步者选项，这会导致某种错误。

无法在本地运行python MRJob

1 个答案: