Question

我想并行运行不同的函数（foo，bar和baz），然后按照调用它们的顺序获取每个函数返回的值。

我尝试过类似下面的代码，但是出于某种原因，它会在Windows上永远循环。是否可以使用例如multiprocessing来实现此目的？请注意，期望函数将参数作为参数（即数据）。

注意：下面的代码适用于Python 3，我想要一个Python 2.7的等价物，因为它是我需要使用的。

from concurrent.futures import ProcessPoolExecutor
from operator import itemgetter

def foo(data):
    return "foo"

def bar(data):
    return "bar"

def baz(data):
    return "baz"

work = [foo, bar, baz]
data = [1,2,3,4,5]

results = []

with ProcessPoolExecutor(max_workers=4) as pool:
    for i, work_item in enumerate(work):
        future = pool.submit(work_item, data)
        def callback(ret):
            results.append((i, ret.result()))
        future.add_done_callback(callback)

results.sort(key=itemgetter(0))
print(results)

Answer 1

我想改变了你的回调方法并达到了你想要的效果。检查结果：

...
with ProcessPoolExecutor(max_workers=4) as pool:
    for i, work_item in enumerate(work):
        future = pool.submit(work_item, data)
        def callback(i):
            def wrapper(ret):
                print(ret, i)
                results.append((i, ret.result()))
            return wrapper
        future.add_done_callback(callback(i))
...

，结果如下：

<Future at 0x1057e4d30 state=finished returned str> 0
<Future at 0x1057e4a58 state=finished returned str> 2
<Future at 0x105779240 state=finished returned str> 1
[(0, 'foo'), (1, 'bar'), (2, 'baz')]

我使用python3在mac os x上运行此代码。我希望它有所帮助。

对于python 2.7，你可以试试这样的东西：

from multiprocessing import Process, Manager

def foo(data, i, results):
    results[i] = "foo"

def bar(data, i, results):
    results[i] = "bar"

def baz(data, i, results):
    results[i] = "baz"

work = [foo, bar, baz]
data = [1,2,3,4,5]

processes = []
results = Manager().dict()
for i, w in enumerate(work):
    p = Process(target=w, args=(data, i, results))
    processes.append(p)
    p.start()

for p in processes:
    p.join()

print results

结果是：

{0: 'foo', 1: 'bar', 2: 'baz'}

How can I recover the return value of a function passed to multiprocessing.Process?

并行运行不同的函数并按特定顺序获取返回的值

1 个答案: