pathos多处理不能发泡

时间:2015-10-08 06:58:44

标签: python multithreading python-2.7 pickle pathos

我遇到类似的问题person。我无法在pathos模块中运行简单的多处理例程并收到酸洗错误。下面是代码和错误。

from pathos.multiprocessing import ProcessingPool
import dill
class ProcClass(object):
    def __init__(self):
        pass
    def f(self,x):
        return x*x
pc = ProcClass()
pl = ProcessingPool(3)
print pl.map(pc.f, range(10))

返回错误:

Exception in thread Thread-2:
Traceback (most recent call last):
  File "/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/threading.py", line 504, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/multiprocessing/pool.py", line 320, in _handle_tasks
    put(task)
PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

我安装了:pp,dill,pox和multiprocessing然后安装了pathos。安装有效,但总是给我这个错误:

警告:以下某个依赖项尚未解决:     pp(ft)> = 1.6.4.5     莳萝&gt; = 0.2.4     pox> = 0.2.2     (多)处理

根据悲剧作者对类似问题的回复,看起来安装存在问题。我已经多次删除并重新安装,每次都验证安装了正确的依赖项。我在MacOS上运行并使用python 2.7。 任何帮助将不胜感激!

1 个答案:

答案 0 :(得分:0)

您似乎错过了一个重要的依赖关系。使用pathos时,您应该使用multiprocess,这是使用multiprocessing序列化程序的dill的分支。

>>> import multiprocessing
>>> multiprocessing.__version__
'0.70a1'
>>> import multiprocess   
>>> multiprocess.__version__
'0.70.4.dev0'
>>> 
>>> multiprocess.Pool().map(lambda x:x*x, range(10))
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
>>>