使用Pool()进行Python多处理

时间:2019-03-09 10:43:25

标签: python multiprocessing

我似乎无法正常工作,我正在尝试使用多处理功能为列表的所有组合调用我的评估函数;

import itertools
from multiprocessing import Pool

def combinations(inputs):
    pool = Pool()
    combi = list(itertools.combinations(inputs, 5))
    outputs = pool.map(evaluate, combi)
    return outputs

def evaluate(input):
    <do stuff>
    return output

a = [[2,2],[4,3],[5,1],[6,3],[7,2],[12,4],[12,2]]
b = combinations(a)

如果我将其放入for循环并一次处理每个循环,则此方法有效。只是pool.map()以及如何传递我不知道的每个列表项。

2 个答案:

答案 0 :(得分:1)

运行:

import multiprocessing

output=[]

a = [[2,2],[4,3],[5,1],[6,3],[7,2],[12,4],[12,2]]


def evaluate(x):
    output=x[0]+x[1]
    return output

def combinations():
    pool = multiprocessing.Pool(64)
    outputs=pool.map(evaluate, a)
    return outputs

if __name__ == '__main__':
    output.append(combinations())

output

输出:

#Out[99]: [[4, 7, 6, 9, 9, 16, 14]]

答案 1 :(得分:1)

以下是21种组合的原始列表:

import itertools
from multiprocessing import Pool
import os

def combinations(inputs):
    combi = list(itertools.combinations(inputs, 5))
    pool = Pool(len(combi))
    outputs = pool.map(evaluate, combi)
    return outputs


def evaluate(input):
    return ['processed by {0}'.format(os.getpid()), input]


a = [[2,2],[4,3],[5,1],[6,3],[7,2],[12,4],[12,2]]
b = combinations(a)
for i in b:
    print(i)

输出:

['processed by 5185', ([2, 2], [4, 3], [5, 1], [6, 3], [7, 2])]
['processed by 5186', ([2, 2], [4, 3], [5, 1], [6, 3], [12, 4])]
['processed by 5187', ([2, 2], [4, 3], [5, 1], [6, 3], [12, 2])]
['processed by 5188', ([2, 2], [4, 3], [5, 1], [7, 2], [12, 4])]
['processed by 5189', ([2, 2], [4, 3], [5, 1], [7, 2], [12, 2])]
['processed by 5190', ([2, 2], [4, 3], [5, 1], [12, 4], [12, 2])]
['processed by 5191', ([2, 2], [4, 3], [6, 3], [7, 2], [12, 4])]
['processed by 5192', ([2, 2], [4, 3], [6, 3], [7, 2], [12, 2])]
['processed by 5193', ([2, 2], [4, 3], [6, 3], [12, 4], [12, 2])]
['processed by 5194', ([2, 2], [4, 3], [7, 2], [12, 4], [12, 2])]
['processed by 5195', ([2, 2], [5, 1], [6, 3], [7, 2], [12, 4])]
['processed by 5196', ([2, 2], [5, 1], [6, 3], [7, 2], [12, 2])]
['processed by 5197', ([2, 2], [5, 1], [6, 3], [12, 4], [12, 2])]
['processed by 5198', ([2, 2], [5, 1], [7, 2], [12, 4], [12, 2])]
['processed by 5199', ([2, 2], [6, 3], [7, 2], [12, 4], [12, 2])]
['processed by 5200', ([4, 3], [5, 1], [6, 3], [7, 2], [12, 4])]
['processed by 5201', ([4, 3], [5, 1], [6, 3], [7, 2], [12, 2])]
['processed by 5202', ([4, 3], [5, 1], [6, 3], [12, 4], [12, 2])]
['processed by 5203', ([4, 3], [5, 1], [7, 2], [12, 4], [12, 2])]
['processed by 5204', ([4, 3], [6, 3], [7, 2], [12, 4], [12, 2])]
['processed by 5205', ([5, 1], [6, 3], [7, 2], [12, 4], [12, 2])]