Question

我想根据一千个视频进行一些异常检测。我已经提取了所有视频的所有帧的功能（使用VGG16）。现在，我将所有内容保存在与每个视频相对应的几个文件中。

当我从磁盘加载文件时，我得到一个形状为np.ndarray的形状（nb_frames，25088）。展平后的25088组件对应于VGGNet16的输出（VGG16输出：1x7x7x512）。

我想用 K 帧填充LSTM K 帧。但是，距我尝试已经过去了好几天，但现在我绝望了，无法使其正常工作...

self.model = Sequential()
# LSTM needs 3 dimensional data (nb_samples, timesteps, input_dim)
self.model.add(CuDNNLSTM(32, return_sequences=True, batch_input_shape=(BATCH_SIZE, SIZE_WINDOW, 25088)))
self.model.add(Dropout(0.2))
self.model.add(Dense(1, activation='softmax'))
self.model.compile(loss='binary_crossentropy', optimizer="rmsprop", metrics=['accuracy'])
self.model.summary()

for (X_train, y_train) in self.batch_generator():
    self.model.fit(X_train, y_train, epochs=10)

这是我的发电机：

def batch_generator(self):
    # for all feature extracted files
    for video in self.videos:
        # videos[0] contains the path to the file
        # videos[1] contains the target (abnormal or not)
        x_train = np.load(video[0])  # load the video's features from disk

        nb_frames = x_train.shape[0]
        data = x_train.shape[1]

        # I've seen on stackoverflow I have to do that...
        x_train = x_train.reshape(nb_frames, data, 1)

        # The target is defined at video level, not frame level, then the same y is applied for all frame of
        # current video
        y_train = np.array([video[1]] * nb_frames)

        # the output shape (the output *shape* is 2 dimensional according to someone on stackoverflow)
        y_train = y_train.reshape(y_train.shape[0], 1)

        nb_windows = len(x_train) // SIZE_WINDOW

        for window_index in range(0, nb_windows):
            start = window_index * SIZE_WINDOW
            end = (window_index + 1) * SIZE_WINDOW
            yield x_train[start:end], y_train[start:end]

我得到了错误：

ValueError: Error when checking input: expected cu_dnnlstm_input 
to have shape (30, 25088) but got array with shape (25088, 1)

30是我要在LSTM中处理的帧数。

此外，每当我尝试更改组件的顺序时，都会收到相同的错误，但是具有不同的值...

编辑：如果我使用第一个答案的解决方案，这是我的代码。但这给了我一个ValueError，无法重塑：

        for window_index in range(0, nb_windows):
            start = window_index * SIZE_WINDOW
            end = (window_index + 1) * SIZE_WINDOW

            chunk = np.array(x_train[start:end])
            chunk = chunk.reshape(int(nb_frames / SIZE_WINDOW), SIZE_WINDOW, data)

            yield chunk, y_train[start:end]

即使我在这里这样做，错误仍然存在：

        [...]
        # I've seen on stackoverflow I have to do that...
        # x_train = x_train.reshape(nb_frames, data, 1)
        x_train = x_train.reshape(int(nb_frames / SIZE_WINDOW), SIZE_WINDOW, data)
        [...]

Answer 1

更改形状：

{9,13}

对不起，我不好

如何在LSTM中提供视频的特征提取帧？

1 个答案: