Python multiprocessing.Process OSError:[Errno 24]打开的文件太多

时间:2020-04-01 02:35:41

标签: python multithreading multiprocessing

我有以下代码:

def formatGravities(gravities):
    # create a list to keep all processes
    processes = []

    # create a list to keep connections
    parent_connections = []

    formatted_gravities = []

    # create a process per instance
    for gravity in gravities:
        # create a pipe for communication
        parent_conn, child_conn = Pipe()
        parent_connections.append(parent_conn)

        # create the process, pass arguments
        process = Process(target=formatGravity,
                          args=(gravity, child_conn))
        processes.append(process)

    # start all processes
    for process in processes:
        process.start()

    # make sure that all processes have finished
    for process in processes:
        process.join()

    for parent_connection in parent_connections:
        formatted_gravities.append(parent_connection.recv()[0])

    return formatted_gravities

len(gravities)数量巨大。我可以理解,我无法同时打开数百万个进程,这可能就是我得到此错误的原因,但是如何更改我的代码,以便在已经有最大进程数的情况下等待生成进程产生。

我确实要求我不能使用multiprocessing.Queue或multiprocessing.Pool。

0 个答案:

没有答案