使用池进行python多重处理

时间:2020-04-21 10:49:48

标签: python multiprocessing pool

我有一份用Python进行的课程作业。

问题是这个:

您将编写一个创建多个进程的程序(p = 10)。每个过程都会收到 整数(y),然后将y乘以4(y * 4)。 y值的集合是一个列表 整数[0,1,2,3,4,5]。您必须使用进程池,因此需要映射这10个 取y个整数列表的过程。请执行以下操作:

您的程序应打印出以下三个。 (1)打印原始清单。 (2)以随机顺序打印每个正在运行的进程名称及其输出y * 4 执行

(1)的输出: 输入列表:[0、1、2、3、4、5]

(2)的输出: 随机输出:

PoolWorker-10输出:[0、4、8、12、16、20]

PoolWorker-11输出:[0、4、8、12、16、20]

PoolWorker-12输出:[0、4、8、12、16、20]

我的第一次尝试是:

import time
from multiprocessing import Pool
from multiprocessing import Process, current_process
def f(number):
    result=4*number
    Process_name=current_process().name
    print(f"Poolworker -{Process_name} output: {result} ")

    return result

if __name__=='__main__':
    p= Pool(processes=10)
    numbers=[0,1,2,3,4,5]
    result=p.map(f,numbers)

输出为:

Poolworker -SpawnPoolWorker-2 output: 0 
Poolworker -SpawnPoolWorker-2 output: 4 
Poolworker -SpawnPoolWorker-2 output: 8 
Poolworker -SpawnPoolWorker-2 output: 12 
Poolworker -SpawnPoolWorker-2 output: 16 
Poolworker -SpawnPoolWorker-2 output: 20 

第二次尝试:

import time
from multiprocessing import Pool
from multiprocessing import Process, current_process
def f(*number):
	numbers=list(number)
	i=0
	for x in numbers:
		numbers[i]=4*numbers[i]
		i+=1
	Process_name=current_process().name
	print(f"Poolworker -{Process_name}output: {numbers} ")

	return numbers

if __name__ == '__main__':
		array=[0,1,2,3,4,5]
		p=Pool(processes=10)
	
		result=p.map(f,array)
		
               

我的更改仍然输出错误。

如何获得所需的输出?

2 个答案:

答案 0 :(得分:2)

问题是f应用于列表中的每个编号,而您希望每个工作人员都在整个列表中进行计算。为了获得这些结果,对代码进行了一些小的修改。

from itertools import repeat
from multiprocessing import Pool
from multiprocessing import Process, current_process

def f(numbers):
    result= [4 * num for num in numbers]
    name=current_process().name
    print(f"Poolworker - {name} output: {result} ")
    return result

if __name__=='__main__':
    WORKERS = 10
    p = Pool(processes=WORKERS)
    numbers = [0, 1, 2, 3, 4, 5]
    print(f"Original - {numbers}")
    result = p.map(f, repeat(numbers, WORKERS))

这将返回以下内容:

Original - [0, 1, 2, 3, 4, 5]
Poolworker - ForkPoolWorker-1 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-2 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-3 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-4 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-5 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-6 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-7 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-8 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-9 output: [0, 4, 8, 12, 16, 20] 
Poolworker - ForkPoolWorker-10 output: [0, 4, 8, 12, 16, 20]

答案 1 :(得分:1)

在您的描述中,您希望所有进程将数组乘以4。

因此,使用并行处理,每个并行处理都被赋予一个数字数组以相乘。

Reference

代码

from multiprocessing import Process

import multiprocessing as mp

def f(numbers, output):
    " Places process name and numbers multiplied by 4 into output queue "
    output.put((mp.current_process().name, [4*n for n in numbers]))

# Define an output queue
output = mp.Queue()

# Setup a list of processes that we want to run
numbers = [1, 2, 3, 4, 5]
processes = [mp.Process(target=f, args=([1, 2, 3, 4, 5], output)) for x in range(10)]

# Run processes
for p in processes:
    p.start()

# Exit the completed processes
for p in processes:
    p.join()

# Get process results from the output queue
results = [output.get() for p in processes]


for id, values in results:
  print(f'PoolWorker {id} output: {values}')

输出

PoolWorker Process-1 output: [4, 8, 12, 16, 20]
PoolWorker Process-3 output: [4, 8, 12, 16, 20]
PoolWorker Process-9 output: [4, 8, 12, 16, 20]
PoolWorker Process-6 output: [4, 8, 12, 16, 20]
PoolWorker Process-7 output: [4, 8, 12, 16, 20]
PoolWorker Process-5 output: [4, 8, 12, 16, 20]
PoolWorker Process-4 output: [4, 8, 12, 16, 20]
PoolWorker Process-8 output: [4, 8, 12, 16, 20]
PoolWorker Process-2 output: [4, 8, 12, 16, 20]
PoolWorker Process-10 output: [4, 8, 12, 16, 20]