Question

我正在尝试编写一些程序来探索使用库多处理的处理优势。但是，当传入multiprocessing.poolmap（，<2Darray>）时，我收到IndexError：元组索引超出范围。我的代码如下

我试图弄平数组，但是没有用

import numpy as np 
import time 
import concurrent.futures 
import multiprocessing

def mean_py(array):
    start_time = time.time() 
    x = array.shape[1] 
    y = array.shape[2] 
    values = np.empty((x,y), type(array[0][0][0]))
    for i in range(x):
        for j in range(y): 
            values[i][j] = ((np.mean(array[:,i,j]))) 
    end_time = time.time() 
    hours, rem = divmod(end_time-start_time, 3600)
    minutes, seconds = divmod(rem,60) 
    print("{:0>2}:{:0>2}:{:05.2f}".format(int(hours), int(minutes), int(seconds)))

    print(f"{'.'*80}")
    return values 

def generate_array():
    a = np.random.randn(1_000_000).reshape(1000,1000)
    b = np.random.randn(1_000_000).reshape(1000,1000)
    c = np.random.randn(1_000_000).reshape(1000,1000)
    d = np.random.randn(1_000_000).reshape(1000,1000)
    e = np.random.randn(1_000_000).reshape(1000,1000)
    f = np.random.randn(1_000_000).reshape(1000,1000)
    g = np.random.randn(1_000_000).reshape(1000,1000)
    h = np.random.randn(1_000_000).reshape(1000,1000)
    i = np.random.randn(1_000_000).reshape(1000,1000)

    arrays = [a, b, c, d, e, f, g, h, i]
    final_array = []
    for array in arrays:
        final_array.append(array)
        print(f"{array} added.")

    final_array = np.asarray(final_array)
    return final_array


start = time.time() 

final_array = generate_array()


pool = multiprocessing.Pool(processes = 2)

result = pool.map(mean_py, final_array)

#result = mean_py(final_array)

end = time.time() 

print(f'\nTime complete: {end-start:.2f}s\n')```



```Traceback (most recent call last):
  File "implementation_v02.py", line 51, in <module>
    result = pool.map(mean_py, final_array)
  File "/home/roger/anaconda3/lib/python3.7/multiprocessing/pool.py", line 290, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/home/roger/anaconda3/lib/python3.7/multiprocessing/pool.py", line 683, in get
    raise self._value
IndexError: tuple index out of range```

Answer 1

这将起作用：

result = pool.map(mean_py, [final_array, final_array])

您需要传递一个列表或一维数组。它尝试映射列表或1d数组中的每个元素，并将其“分配”到您的pool.map函数参数（mean_py）。它不知道如何处理传递给它的3维数组...它应该占用9 * 1000 * 1000个元素中的每个元素吗？切片并仅通过一维？哪一个？

IndexError：尝试在multiprocessing.Pool中传递2d数组时，元组索引超出范围

1 个答案: