Python类方法并行化

时间:2015-05-11 11:22:26

标签: python parallel-processing

我需要提升我的python应用程序。该解决方案应该是微不足道的:

import time
from multiprocessing import Pool


class A:
    def method1(self):
        time.sleep(1)
        print('method1')
        return 'method1'

    def method2(self):
        time.sleep(1)
        print('method2')
        return 'method2'

    def method3(self):
        pool = Pool()
        time1 = time.time()
        res1 = pool.apply_async(self.method1, [])
        res2 = pool.apply_async(self.method2, [])
        res1 = res1.get()
        res2 = res2.get()
        time2 = time.time()
        print('res1 = {0}'.format(res1))
        print('res2 = {0}'.format(res2))
        print('time = {0}'.format(time2 - time1))



a = A()
a.method3()

但是每次我启动这个简单的程序时都会遇到异常:

Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python3.2/threading.py", line 740, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.2/threading.py", line 693, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.2/multiprocessing/pool.py", line 346, in _handle_tasks
    put(task)
_pickle.PicklingError: Can't pickle <class 'method'>: attribute lookup builtins.method failed

似乎python无法并行化类方法。 我一整天都在拼命搜索,但我仍然不明白如何解决这个问题(可能我用谷歌搜索不够好)。 Python多处理文档似乎很差。

我不想破坏我的类,将它与全局方法分开。 以下代码似乎可行:

class B:
    def method1(self):
        time.sleep(1)
        print "B.method1"
        return "B.method1"

    def method2(self):
        time.sleep(1)
        print "B.method2"
        return "B.method2"

def method1(b):
    time.sleep(1)
    print('method1')
    return  b.method1()


def method2(b):
    time.sleep(1)
    print('method2')
    return  b.method2()


def method3():
    pool = Pool()
    time1 = time.time()
    b = B()
    res1 = pool.apply_async(method1, [b])
    res2 = pool.apply_async(method2, [b])
    res1 = res1.get()
    res2 = res2.get()
    time2 = time.time()
    print('res1 = {0}'.format(res1))
    print('res2 = {0}'.format(res2))
    print('time = {0}'.format(time2 - time1))

method3()

但我仍然不了解如何在类方法中进行python并行化工作。有人可以帮我解决这个障碍吗? 可能还有其他的并行化方法我不知道在这种情况下可以应用吗?我需要一个可行的代码作为例子。任何帮助将不胜感激。

0 个答案:

没有答案