如何基于每行的起始索引和终止索引有效地索引numpy数组

Question

我有一个二维的numpy数组，其中的行是要素的时间序列，基于该数组我正在训练神经网络。为了通用起见，我想在随机点上对这些时间序列进行子集化。我希望它们也具有最小子集长度。但是，网络需要固定长度的时间序列，因此我需要用零预先填充结果子集。

当前，我正在使用下面的代码来完成此操作，其中包括讨厌的for循环，因为我不知道如何针对这个特定问题使用花式索引。由于这段代码是网络数据生成器的一部分，因此它需要快速以跟上需要大量数据的GPU的步伐。没有人知道在没有for循环的情况下执行此操作的方法吗？

import numpy as np
import matplotlib.pyplot as plt

# Amount of time series to consider
batchsize = 25

# Original length of the time series
timesteps = 150

# As an example, fill the 2D array with sine function time series
sinefunction = np.expand_dims(np.sin(np.arange(timesteps)), axis=0)
originalarray = np.repeat(sinefunction, batchsize, axis=0)

# Now the real thing, we want:
# - to start the time series at a random moment (between 0 and maxstart)
# - to end the time series at a random moment
# - however with a minimum length of the resulting subset time series (minlength)
maxstart = 50
minlength = 75

# get random starts
randomstarts = np.random.choice(np.arange(0, maxstart), size=batchsize)

# get random stops
randomstops = np.random.choice(np.arange(maxstart + minlength, timesteps), size=batchsize)

# determine the resulting random sizes of the subset time series
randomsizes = randomstops - randomstarts

# finally create a new 2D array with all the randomly subset time series, however pre-padded with zeros
# THIS IS THE FOR LOOP WE SHOULD TRY TO AVOID
cutarray = np.zeros_like(originalarray)
for i in range(batchsize):
    cutarray[i, -randomsizes[i]:] = originalarray[i, randomstarts[i]:randomstops[i]]

要显示该函数的进出内容：

# Show that it worked
f, ax = plt.subplots(2, 1)
ax[0].imshow(originalarray)
ax[0].set_title('original array')
ax[1].imshow(cutarray)
ax[1].set_title('zero-padded subset array')

Answer 1

方法1：基于视图的

我们可以利用基于np.lib.stride_tricks.as_strided的scikit-image's view_as_windows来将滑动窗口视图转换为输入的零填充版本，并分配为输出的零填充版本。由于衣衫nature的性质，所有这些填充对于矢量化解决方案都是必需的。有利的是，处理视图会提高内存和性能。

实现看起来像这样-

from skimage.util.shape import view_as_windows

n = randomsizes.max()
max_extent = randomstarts.max()+n
padlen = max_extent - origalarray.shape[1]
p = np.zeros((origalarray.shape[0],padlen),dtype=origalarray.dtype)
a = np.hstack((origalarray,p))
w = view_as_windows(a,(1,n))[...,0,:]
out_vals = w[np.arange(len(randomstarts)),randomstarts]

out_starts = origalarray.shape[1]-randomsizes    
out_extensions_max = out_starts.max()+n

out = np.zeros((origalarray.shape[0],out_extensions_max),dtype=origalarray.dtype)
w2 = view_as_windows(out,(1,n))[...,0,:]
w2[np.arange(len(out_starts)),out_starts] = out_vals
cutarray_out = out[:,:origalarray.shape[1]]

方法2：使用masking

cutarray_out = np.zeros_like(origalarray)
r = np.arange(origalarray.shape[1])
m = (randomstarts[:,None]<=r) & (randomstops[:,None]>r)
s = origalarray.shape[1]-randomsizes
m2 = s[:,None]<=r
cutarray_out[m2] = origalarray[m]