Question

举个简单的例子，当我运行以下代码时：

from multiprocessing import Pool

class foo(object):
    def __init__(self):
        self.a = 0

    def g(self, x):
        return x + self.a

    def bar(self):
        def f(x):
            return self.g(x)**2

        # Compute [f(x) for x in range(1000)] in parallel
        pool = Pool()
        res = pool.map(f, range(1000))
        pool.close()
        pool.join()

        return res

res = foo().bar()

我收到错误：

---------------------------------------------------------------------------
PicklingError                             Traceback (most recent call last)
<ipython-input-7-cbbd76310b60> in <module>()
     19         return res
     20
---> 21 res = foo().bar()

<ipython-input-7-cbbd76310b60> in bar(self)
     13         # Compute [f(x) for x in range(1000)] in parallel
     14         pool = Pool()
---> 15         res = pool.map(f, range(1000))
     16         pool.close()
     17         pool.join()

/Users/jiaseny/anaconda2/lib/python2.7/multiprocessing/pool.pyc in map(self, func, iterable, chunksize)
    251         '''
    252         assert self._state == RUN
--> 253         return self.map_async(func, iterable, chunksize).get()
    254
    255     def imap(self, func, iterable, chunksize=1):

/Users/jiaseny/anaconda2/lib/python2.7/multiprocessing/pool.pyc in get(self, timeout)
    570             return self._value
    571         else:
--> 572             raise self._value
    573
    574     def _set(self, i, obj):

PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

我相信它是因为函数f在计算过程中调用self.g。有什么方法可以解决这个问题吗？

一种解决方法是在g中使bar成为本地函数，但在更复杂的类中，这将是不切实际的，因为g可能还包含一些其他函数{{ 1}}依旧......

Python多处理中的PicklingError

0 个答案: