Question

我有一个嵌套列表（基本上是一个矩阵）的参数，我想传递给一个耗时的函数。我想使用multiprocess.Pool并行化代码。如果我有一个参数列表，我只需在该列表上调用Pool.map。

但是如何在矩阵上使用Pool.map以便结果仍然是2D矩阵格式？

假设我有一个函数f(x)和一个列表

l = [[1, 2, ..., n], [n + 1, ..., 2n], [(m-1)*n, ..., n*m]

如何获取列表

[[f(1), f(2), ..., f(n)], [f(n + 1), ..., f(2*n)], [f((m-1)*n), ..., f(n*m)]

通过使用进程池来并行化f的评估，这是一个耗时的函数？

Answer 1

这里有一个简单的示例，只需在应用map的矩阵上调用pool.map：

import multiprocessing


def add(x):
    return x + 10


if __name__ == "__main__":

    pool = multiprocessing.Pool() # build the process pool
    l = [range(10) for _ in xrange(10)] # we build an example list
    res = map(lambda x: pool.map(add, x), l) # we apply the pool.map with the map to the list
    multiprocessing.freeze_support() # this line is needed on windows only
    print res

这是印刷品：

>>>[[10, 11, 12, 13, 14, 15, 16, 17, 18, 19], [10, 11, 12, 13, 14, 15, 16, 17, 18, 19], [10, 11, 12, 13, 14, 15, 16, 17, 18, 19], [10, 11, 12, 13, 14, 15, 16, 17, 18, 19], [10, 11, 12, 13, 14, 15, 16, 17, 18, 19], [10, 11, 12, 13, 14, 15, 16, 17, 18, 19], [10, 11, 12, 13, 14, 15, 16, 17, 18, 19], [10, 11, 12, 13, 14, 15, 16, 17, 18, 19], [10, 11, 12, 13, 14, 15, 16, 17, 18, 19], [10, 11, 12, 13, 14, 15, 16, 17, 18, 19]]

如何将并行Pool.map应用于嵌套列表

1 个答案: