我使用Python 2.7运行Windows 7。
我正在尝试使用multiprocessing
模块运行基本的多处理代码来测试可用性。我见过类似的错误here和here,但下面的代码没有这些问题。
当我从Anaconda命令提示符或Spyder IDE Python控制台运行以下代码(或使用multiprocessing
模块的任何代码)时,我收到以下错误消息。
当我在Spyder IDE iPython控制台中运行代码时,我没有收到错误,但是程序永远不会完成,并且从我的Windows Task Manager
我可以看到python没有使用任何CPU,即使Spyder显示它仍在运行(控制台上方的红色方块为红色)。
import multiprocessing
def funSquare(num):
return num ** 2
if __name__ == '__main__':
pool = multiprocessing.Pool()
results = pool.map(funSquare, range(10))
print(results)
Process PoolWorker-1:
Traceback (most recent call last):
File "C:\Anaconda\lib\multiprocessing\process.py", line 258, in _bootstrap
self.run()
File "C:\Anaconda\lib\multiprocessing\process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "C:\Anaconda\lib\multiprocessing\pool.py", line 102, in worker
task = get()
File "C:\Anaconda\lib\multiprocessing\queues.py", line 376, in get
return recv()
AttributeError: 'module' object has no attribute 'funSquare'
Process PoolWorker-2:
Process PoolWorker-5:
Traceback (most recent call last):
File "C:\Anaconda\lib\multiprocessing\process.py", line 258, in _bootstrap
self.run()
File "C:\Anaconda\lib\multiprocessing\process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "C:\Anaconda\lib\multiprocessing\pool.py", line 102, in worker
task = get()
File "C:\Anaconda\lib\multiprocessing\queues.py", line 376, in get
return recv()
AttributeError: 'module' object has no attribute 'funSquare'
Traceback (most recent call last):
File "C:\Anaconda\lib\multiprocessing\process.py", line 258, in _bootstrap
self.run()
File "C:\Anaconda\lib\multiprocessing\process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "C:\Anaconda\lib\multiprocessing\pool.py", line 102, in worker
task = get()
File "C:\Anaconda\lib\multiprocessing\queues.py", line 376, in get
return recv()
AttributeError: 'module' object has no attribute 'funSquare'
Process PoolWorker-4:
Traceback (most recent call last):
File "C:\Anaconda\lib\multiprocessing\process.py", line 258, in _bootstrap
self.run()
File "C:\Anaconda\lib\multiprocessing\process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "C:\Anaconda\lib\multiprocessing\pool.py", line 102, in worker
task = get()
File "C:\Anaconda\lib\multiprocessing\queues.py", line 376, in get
return recv()
AttributeError: 'module' object has no attribute 'funSquare'
Process PoolWorker-3:
Traceback (most recent call last):
File "C:\Anaconda\lib\multiprocessing\process.py", line 258, in _bootstrap
self.run()
File "C:\Anaconda\lib\multiprocessing\process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "C:\Anaconda\lib\multiprocessing\pool.py", line 102, in worker
task = get()
File "C:\Anaconda\lib\multiprocessing\queues.py", line 376, in get
return recv()
AttributeError: 'module' object has no attribute 'funSquare'
Process PoolWorker-6:
Traceback (most recent call last):
File "C:\Anaconda\lib\multiprocessing\process.py", line 258, in _bootstrap
self.run()
File "C:\Anaconda\lib\multiprocessing\process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "C:\Anaconda\lib\multiprocessing\pool.py", line 102, in worker
task = get()
File "C:\Anaconda\lib\multiprocessing\queues.py", line 376, in get
return recv()
AttributeError: 'module' object has no attribute 'funSquare'
Process PoolWorker-8:
Process PoolWorker-7:
Traceback (most recent call last):
File "C:\Anaconda\lib\multiprocessing\process.py", line 258, in _bootstrap
self.run()
File "C:\Anaconda\lib\multiprocessing\process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "C:\Anaconda\lib\multiprocessing\pool.py", line 102, in worker
task = get()
File "C:\Anaconda\lib\multiprocessing\queues.py", line 376, in get
return recv()
AttributeError: 'module' object has no attribute 'funSquare'
Traceback (most recent call last):
File "C:\Anaconda\lib\multiprocessing\process.py", line 258, in _bootstrap
self.run()
File "C:\Anaconda\lib\multiprocessing\process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "C:\Anaconda\lib\multiprocessing\pool.py", line 102, in worker
task = get()
File "C:\Anaconda\lib\multiprocessing\queues.py", line 376, in get
return recv()
AttributeError: 'module' object has no attribute 'funSquare'
Process PoolWorker-9:
Traceback (most recent call last):
File "C:\Anaconda\lib\multiprocessing\process.py", line 258, in _bootstrap
self.run()
File "C:\Anaconda\lib\multiprocessing\process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "C:\Anaconda\lib\multiprocessing\pool.py", line 102, in worker
task = get()
File "C:\Anaconda\lib\multiprocessing\queues.py", line 376, in get
return recv()
AttributeError: 'module' object has no attribute 'funSquare'
Process PoolWorker-10:
Traceback (most recent call last):
File "C:\Anaconda\lib\multiprocessing\process.py", line 258, in _bootstrap
self.run()
File "C:\Anaconda\lib\multiprocessing\process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "C:\Anaconda\lib\multiprocessing\pool.py", line 102, in worker
task = get()
File "C:\Anaconda\lib\multiprocessing\queues.py", line 376, in get
return recv()
AttributeError: 'module' object has no attribute 'funSquare'
有人可以告诉我如何解决这个问题吗?如果需要更多细节,请告诉我。谢谢!
答案 0 :(得分:1)
我正在运行python 3并且IDE在anaconda(windows)中是spyder所以我也得到了错误。但经过漫长的一天搜索,我得到了一些解决方案,它帮助我运行相同的代码我的Windows机器。这个网站帮助我找到了解决方案:
http://python.6.x6.nabble.com/Multiprocessing-Pool-woes-td5047050.html
因为我使用的是python 3,所以我改变了这个程序:
from types import FunctionType
import marshal
def _applicable(*args, **kwargs):
name = kwargs['__pw_name']
code = marshal.loads(kwargs['__pw_code'])
gbls = globals() #gbls = marshal.loads(kwargs['__pw_gbls'])
defs = marshal.loads(kwargs['__pw_defs'])
clsr = marshal.loads(kwargs['__pw_clsr'])
fdct = marshal.loads(kwargs['__pw_fdct'])
func = FunctionType(code, gbls, name, defs, clsr)
func.fdct = fdct
del kwargs['__pw_name']
del kwargs['__pw_code']
del kwargs['__pw_defs']
del kwargs['__pw_clsr']
del kwargs['__pw_fdct']
return func(*args, **kwargs)
def make_applicable(f, *args, **kwargs):
if not isinstance(f, FunctionType): raise ValueError('argument must be a function')
kwargs['__pw_name'] = f.__name__ # edited
kwargs['__pw_code'] = marshal.dumps(f.__code__) # edited
kwargs['__pw_defs'] = marshal.dumps(f.__defaults__) # edited
kwargs['__pw_clsr'] = marshal.dumps(f.__closure__) # edited
kwargs['__pw_fdct'] = marshal.dumps(f.__dict__) # edited
return _applicable, args, kwargs
def _mappable(x):
x,name,code,defs,clsr,fdct = x
code = marshal.loads(code)
gbls = globals() #gbls = marshal.loads(gbls)
defs = marshal.loads(defs)
clsr = marshal.loads(clsr)
fdct = marshal.loads(fdct)
func = FunctionType(code, gbls, name, defs, clsr)
func.fdct = fdct
return func(x)
def make_mappable(f, iterable):
if not isinstance(f, FunctionType): raise ValueError('argument must be a function')
name = f.__name__ # edited
code = marshal.dumps(f.__code__) # edited
defs = marshal.dumps(f.__defaults__) # edited
clsr = marshal.dumps(f.__closure__) # edited
fdct = marshal.dumps(f.__dict__) # edited
return _mappable, ((i,name,code,defs,clsr,fdct) for i in iterable)
在此功能之后,上面的问题代码也改变了一点:
from multiprocessing import Pool
from poolable import make_applicable, make_mappable
def cube(x):
return x**3
if __name__ == "__main__":
pool = Pool(processes=2)
results = [pool.apply_async(*make_applicable(cube,x)) for x in range(1,7)]
print([result.get(timeout=10) for result in results])
我得到了输出:
[1, 8, 27, 64, 125, 216]
我认为这篇文章可能对某些Windows用户有用。