Question

我使用concurrent.futures.ProcessPoolExecutor()写了一个简单的代码，您可以在下面看到。我正在Core-i7笔记本电脑上的Windows 10（64位）上使用Python 3.7.4。

import time
import concurrent.futures

def f(x):
    lo = 0
    for i in range(x):
        lo += i
    return(lo)

n = 7

if __name__ == '__main__':
    t1 = time.perf_counter()
    with concurrent.futures.ProcessPoolExecutor() as executor:
        Ans = [executor.submit(f, 10**7-i) for i in range(n)]

        for f in concurrent.futures.as_completed(Ans):
            print(f.result())

    t2 = time.perf_counter()
    print('completed at', t2-t1, 'seconds')

变量n确定要执行多少个进程。当我将n设置为1、2、4、7时，一切正常。例如，n=7的输出是

49999995000000
49999955000010
49999965000006
49999985000001
49999975000003
49999945000015
49999935000021
completed at 2.0607623 seconds

但是对于n=10，它会出现以下错误！

49999945000015
49999955000010
49999965000006
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
  File "E:\Python37\lib\multiprocessing\queues.py", line 236, in _feed
    obj = _ForkingPickler.dumps(obj)
  File "E:\Python37\lib\multiprocessing\reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function f at 0x00000285BFC4E0D8>: it's not the same object as __main__.f
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "e:/Python37/Python files/Parallel struggle/Python_20191219_parallel_3.py", line 23, in <module>
    print(f.result())
  File "E:\Python37\lib\concurrent\futures\_base.py", line 428, in result
    return self.__get_result()
  File "E:\Python37\lib\concurrent\futures\_base.py", line 384, in __get_result
    raise self._exception
  File "E:\Python37\lib\multiprocessing\queues.py", line 236, in _feed
    obj = _ForkingPickler.dumps(obj)
  File "E:\Python37\lib\multiprocessing\reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function f at 0x00000285BFC4E0D8>: it's not the same object as __main__.f

如果代码有问题，为什么某些过程完成了？发生了什么错误？它是特定于Windows上的python吗？是关于CPU的数量吗？

concurrent.futures.ProcessPoolExecutor（）对进程数是否有任何限制？

0 个答案: