Question

pyhton ProcessPoolExecutor在公用行中工作，但添加到函数后无法运行

它是这样工作的

from concurrent import futures

def multi_process(func, paras, threads):
    with futures.ProcessPoolExecutor(max_workers=threads) as pool:
        res = pool.map(func, paras, chunksize=threads)
    return list(res)
p = multi_process(func,paras,threads)

但不能完全正常工作

def upper(paras,threads):
    def func:
        some func
    def multi_process(func, paras, threads):
        with futures.ProcessPoolExecutor(max_workers=threads) as pool:
            res = pool.map(func, paras, chunksize=threads)
        return list(res)
    p = multi_process(func,paras,threads)
    return p
p = upper(paras,threads)

没有警告或错误，但很长时间没有任何响应。

Answer 1

您确实得到了一个错误。它的。

AttributeError: Can't pickle local object 'upper.<locals>.func'.

原因是要使多处理工作，它需要在全局级别定义功能。

要实现您想要的目标，可以执行以下操作：

from concurrent import futures

# Has to be a global function
def func(para):
    print(para)


def upper(paras,threads):
    # This cannot be a local function.
    #def func(para):
    #    print(para)
    def multi_process(func, paras, threads):
        with futures.ProcessPoolExecutor(max_workers=threads) as pool:
            res = pool.map(func, paras, chunksize=threads)
        return list(res)
    p = multi_process(func, paras, threads)
    return p

paras = [1, 2, 3]
threads = 3
p = upper(paras,threads)

python ProcessPoolExecutor在起作用时不起作用

1 个答案: