cProfiler与多处理奇怪地工作

时间:2018-12-15 08:27:35

标签: python multiprocessing profiler pathos

我收到此代码错误:

http://www.your-domain.com/index.html

当我与from pathos.multiprocessing import ProcessingPool def diePlz(im): print('Whoopdepoop!') def caller(): im = 1 pool = ProcessingPool() pool.map(diePlz,[im,im,im,im]) if __name__=='__main__': caller() :(cProfiler)一起运行时

python3 -m cProfile testProfiler.py

但是当我在没有multiprocess.pool.RemoteTraceback: """ Traceback (most recent call last): File "/home/rohit/.local/lib/python3.6/site-packages/multiprocess/pool.py", line 119, in worker result = (True, func(*args, **kwds)) File "/home/rohit/.local/lib/python3.6/site-packages/multiprocess/pool.py", line 44, in mapstar return list(map(*args)) File "/home/rohit/.local/lib/python3.6/site-packages/pathos/helpers/mp_helper.py", line 15, in <lambda> func = lambda args: f(*args) File "testProfiler.py", line 3, in diePlz print('Whoopdepoop!') NameError: name 'print' is not defined """ The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/usr/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/usr/lib/python3.6/cProfile.py", line 160, in <module> main() File "/usr/lib/python3.6/cProfile.py", line 153, in main runctx(code, globs, None, options.outfile, options.sort) File "/usr/lib/python3.6/cProfile.py", line 20, in runctx filename, sort) File "/usr/lib/python3.6/profile.py", line 64, in runctx prof.runctx(statement, globals, locals) File "/usr/lib/python3.6/cProfile.py", line 100, in runctx exec(cmd, globals, locals) File "testProfiler.py", line 11, in <module> caller() File "testProfiler.py", line 8, in caller pool.map(diePlz,[im,im,im,im]) File "/home/rohit/.local/lib/python3.6/site-packages/pathos/multiprocessing.py", line 137, in map return _pool.map(star(f), zip(*args)) # chunksize File "/home/rohit/.local/lib/python3.6/site-packages/multiprocess/pool.py", line 266, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/home/rohit/.local/lib/python3.6/site-packages/multiprocess/pool.py", line 644, in get raise self._value NameError: name 'print' is not defined 的情况下运行它时:

cProfiler

我提供的代码是解决该问题的最小示例。我想调试一个更大的代码,但不能这样做,因为$ python3 testProfiler.py Whoopdepoop! Whoopdepoop! Whoopdepoop! Whoopdepoop! 一直在引发奇怪的错误。

在这种情况下,重点是

cProfiler

,这意味着NameError: name 'print' is not defined 无法识别python3本身。在我的代码中,它无法识别print

1 个答案:

答案 0 :(得分:2)

所以,我意识到这是在原始帖子之后很长一段时间,但我遇到了完全相同的问题。

在我的例子中,我得到了与原始帖子完全相同的错误 - python 内置函数,例如 print()len() 导致了这样的错误:

NameError: name 'len' is not defined

我目前正在运行 multiprocess 版本 0.70.11.1 和 dill 版本 0.3.3(使基于进程的并行工作的悲观组件)。

根据我在问题评论中的发现:https://github.com/uqfoundation/pathos/issues/129#issuecomment-536081859 软件包作者之一建议尝试:

import dill
dill.settings['recurse'] = True

至少在我的情况下,以上修复了错误!