用于时间依赖特征的卷积神经网络

时间:2015-07-21 13:59:07

标签: neural-network convolution feature-extraction feature-selection conv-neural-network

我需要从一系列图像中减少维数。更具体地,每个图像是球移动的快照,并且最佳特征将是其位置和速度。据我所知,CNN是减少图像分类功能的最先进技术,但在这种情况下,只提供了一个帧。在不同的时间步长给出许多图像是否有可能提取时间相关的特征?否则这是最先进的技术吗?

这是我第一次使用CNN,我也非常感谢任何参考或任何其他建议。

1 个答案:

答案 0 :(得分:2)

If you want to be able to have the network somehow recognize a progression which is time dependent, you should probably look into recurrent neural nets (RNN). Since you would be operating on video, you should look into recurrent convolutional neural nets (RCNN) such as in: http://jmlr.org/proceedings/papers/v32/pinheiro14.pdf

Recurrence adds some memory of a previous state of the input data. See this good explanation by Karpathy: http://karpathy.github.io/2015/05/21/rnn-effectiveness/

In your case you need to have the recurrence across multiple images instead of just within one image. It would seem like the first problem you need to solve is the image segmentation problem (being able to pick the ball out of the rest of the image) and the first paper linked above deals with segmentation. (then again, maybe you're trying to take advantage of the movement in order to identify the moving object?)

Here's another thought: perhaps you could only look at differences between sequential frames and use that as your input data to your convnet? The input "image" would then show where the moving object was in the previous frame and where it is in the current one. Larger differences would indicate larger amounts of movement. That would probably have a similar effect to using a recurrent network.