Python多进程似乎无法正常工作

时间:2018-04-05 03:13:58

标签: python multiprocessing

我正在学习Python多处理,这是我的代码:

import multiprocessing as mp
import time 

def cube(list_i, result_i):
    for i in list_i:
        result_i[i]=i**3
    return
def test():
    result_i=list(range(10000000))
    list_i=list(range(10000000))
    single=mp.Process(target=cube, args=(range(10000000),result_i))
    t=time.time()
    single.start()
    single.join()
    print(time.time()-t)
    multiple=[mp.Process(target=cube, args=(i, result_i)) for i in [range(i*2500000, i*2500000+2500000) for i in range(4)]]
    t=time.time()
    for process in multiple:
        process.start()
    for process in multiple:
        process.join()
    print(time.time()-t)
    return

if __name__=='__main__':
    test()

输出:

12.0096
32.0467   

似乎单个过程更快?我的电脑的CPU是i5-5200,有4个核心。

1 个答案:

答案 0 :(得分:0)

以下是Manager的示例。

import multiprocessing as mp
import time

import requests


# for example we will fetch data about countries
def call_url(url: str, result: dict):
    response = requests.get(url)

    result['data'] = response.json()


def test():
    manager = mp.Manager()
    # result of process
    result = manager.dict()
    # send url and result to process
    single = mp.Process(
        target=call_url,
        args=(
            'https://restcountries.eu/rest/v2/all',
            result,
        ))
    start_time = time.time()
    single.start()
    # wait when process is finish and print time + result
    single.join()
    print('\nSingle time: %s' % (time.time() - start_time))
    print('\nSingle result: %s' % result.values())

    processes = []
    start_time = time.time()
    # create 4 processes and fetch data in each process
    for i in range(4):
        result = manager.dict()
        process = mp.Process(
            target=call_url,
            args=(
                'https://restcountries.eu/rest/v2/all',
                result,
            ))
        processes.append((process, result, ))
        process.start()
    # wait until all processes are finished
    for process, _ in processes:
        process.join()
    # print results of each process and time
    print('\nresults: ')
    for _, result in processes:
        print('\n')
        print(result.values())

    print('\n4 process time: %s' % (time.time() - start_time))

if __name__ == '__main__':
    test()

运行我们的脚本(Python 3.6.1)。输出示例:

Single time: 0.9279030323028564

Single result: [[{'name': 'Afghanistan'...

results: # long output (the same result as in single - [[{'name': 'Afghanistan'...)

4 process time: 1.0175669193267822

Process finished with exit code 0

所以,你可以看到0.9~ = 1.0。但在第二次发送了4个请求。您也可以使用QueuePool等。有很多关于SO的例子(也在互联网上)。

希望这有帮助。