我正在尝试编写一些程序来探索使用库多处理的处理优势。但是,当传入multiprocessing.poolmap(,<2Darray>)时,我收到IndexError:元组索引超出范围。我的代码如下
我试图弄平数组,但是没有用
import numpy as np
import time
import concurrent.futures
import multiprocessing
def mean_py(array):
start_time = time.time()
x = array.shape[1]
y = array.shape[2]
values = np.empty((x,y), type(array[0][0][0]))
for i in range(x):
for j in range(y):
values[i][j] = ((np.mean(array[:,i,j])))
end_time = time.time()
hours, rem = divmod(end_time-start_time, 3600)
minutes, seconds = divmod(rem,60)
print("{:0>2}:{:0>2}:{:05.2f}".format(int(hours), int(minutes), int(seconds)))
print(f"{'.'*80}")
return values
def generate_array():
a = np.random.randn(1_000_000).reshape(1000,1000)
b = np.random.randn(1_000_000).reshape(1000,1000)
c = np.random.randn(1_000_000).reshape(1000,1000)
d = np.random.randn(1_000_000).reshape(1000,1000)
e = np.random.randn(1_000_000).reshape(1000,1000)
f = np.random.randn(1_000_000).reshape(1000,1000)
g = np.random.randn(1_000_000).reshape(1000,1000)
h = np.random.randn(1_000_000).reshape(1000,1000)
i = np.random.randn(1_000_000).reshape(1000,1000)
arrays = [a, b, c, d, e, f, g, h, i]
final_array = []
for array in arrays:
final_array.append(array)
print(f"{array} added.")
final_array = np.asarray(final_array)
return final_array
start = time.time()
final_array = generate_array()
pool = multiprocessing.Pool(processes = 2)
result = pool.map(mean_py, final_array)
#result = mean_py(final_array)
end = time.time()
print(f'\nTime complete: {end-start:.2f}s\n')```
```Traceback (most recent call last):
File "implementation_v02.py", line 51, in <module>
result = pool.map(mean_py, final_array)
File "/home/roger/anaconda3/lib/python3.7/multiprocessing/pool.py", line 290, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/home/roger/anaconda3/lib/python3.7/multiprocessing/pool.py", line 683, in get
raise self._value
IndexError: tuple index out of range```
答案 0 :(得分:1)
这将起作用:
result = pool.map(mean_py, [final_array, final_array])
您需要传递一个列表或一维数组。它尝试映射列表或1d数组中的每个元素,并将其“分配”到您的pool.map函数参数(mean_py)。它不知道如何处理传递给它的3维数组...它应该占用9 * 1000 * 1000个元素中的每个元素吗?切片并仅通过一维?哪一个?