I want to train a LSTM network which starts at a patch in the image, outputs location where it should move next, and the input at next timestep is the patch around new cell.
Is there a way I could train this using Keras/PyTorch/Caffe/Tensorflow? Traditionally, networks have fixed input-output sequence for training. I need a network to have dynamic input-output sequence.
PS: I don't want to go for reinforcement learning, because I want to model path characteristics for future prediction.