Keras LSTM一次预测1个步骤

时间：2017-09-28 02:06:18

标签： keras lstm

编辑添加： 我发现我认为是一个有效的解决方案：https://bleyddyn.github.io/posts/2017/10/keras-lstm/

我正在尝试使用Conv / LSTM网络来控制机器人。我想我已经完成了所有设置，所以我可以开始对重放内存中的批量数据进行训练，但我无法弄清楚如何实际使用它来控制机器人。简化的测试代码如下。

import numpy as np

from keras.models import Sequential
from keras.layers import Dense, Flatten, Input
from keras.layers import Convolution2D
from keras.layers.recurrent import LSTM
from keras.layers.wrappers import TimeDistributed
from keras.utils import to_categorical

def make_model(num_actions, timesteps, input_dim, l2_reg=0.005 ):
    input_shape=(timesteps,) + input_dim
    model = Sequential()
    model.add(TimeDistributed( Convolution2D(8, (3, 3), strides=(2,2), activation='relu' ), input_shape=input_shape) )
    model.add(TimeDistributed( Convolution2D(16, (3, 3), strides=(2,2), activation='relu', ) ))
    model.add(TimeDistributed( Convolution2D(32, (3, 3), strides=(2,2), activation='relu', ) ))
    model.add(TimeDistributed(Flatten()))
    model.add(LSTM(512, return_sequences=True, activation='relu', unroll=True))
    model.add(Dense(num_actions, activation='softmax', ))
    model.compile(loss='categorical_crossentropy', optimizer='adam' )
    return model

batch_size = 16
timesteps = 10
num_actions = 6
model = make_model( num_actions, timesteps, (84,84,3) )
model.summary()

# Fake training batch. Would be pulled from a replay memory
batch = np.random.uniform( low=0, high=255, size=(batch_size,timesteps,84,84,3) )
y = np.random.randint( 0, high=5, size=(160) )
y = to_categorical( y, num_classes=num_actions )
y = y.reshape( batch_size, timesteps, num_actions )
# stateful should be false here
pred = model.train_on_batch( batch, y )

# move trained network to robot

# This works, but it isn't practical to not get outputs (actions) until after 10 timesteps and I don't think the LSTM internal state would be correct if I tried a rolling queue of input images.
batch = np.random.uniform( low=0, high=255, size=(1,timesteps,84,84,3) )
pred = model.predict( batch, batch_size=1 )

# This is what I would need to do on my robot, with the LSTM keeping state between calls to predict
max_time = 10 # or 100000, or forever, etc.
for i in range(max_time) :
    image = np.random.uniform( low=0, high=255, size=(1,1,84,84,3) ) # pull one image from camera
    # stateful should be true here
    pred = model.predict( image, batch_size=1 )
    # take action based on pred

我在“model.predict（图片...”行）上遇到的错误是：

  

ValueError：检查时出错：预期time_distributed_1_input具有形状（无，10,84,84,3）但是具有形状的数组（1,1,84,84,3）

这是可以理解的，但我无法找到解决办法。 我甚至不知道Keras是否能够正确使用TimeDistributed图层。

那么，这在卡拉斯甚至可能吗？如果是这样，怎么样？

如果没有，是否可以在TF或PyTorch中使用？

感谢您的任何建议！

编辑添加运行代码，虽然它不一定正确。仍然需要在OpenAI健身房任务上进行测试。

import numpy as np

from keras.models import Sequential
from keras.layers import Dense, Flatten, Input
from keras.layers import Convolution2D
from keras.layers.recurrent import LSTM
from keras.layers.wrappers import TimeDistributed
from keras.utils import to_categorical

def make_model(num_actions, timesteps, input_dim, l2_reg=0.005 ):
    input_shape=(1,None) + input_dim
    model = Sequential()
    model.add(TimeDistributed( Convolution2D(8, (3, 3), strides=(2,2), activation='relu' ), batch_input_shape=input_shape) )
    model.add(TimeDistributed( Convolution2D(16, (3, 3), strides=(2,2), activation='relu', ) ))
    model.add(TimeDistributed( Convolution2D(32, (3, 3), strides=(2,2), activation='relu', ) ))
    model.add(TimeDistributed(Flatten()))
    model.add(LSTM(512, return_sequences=True, activation='relu', stateful=True))
    model.add(Dense(num_actions, activation='softmax', ))
    model.compile(loss='categorical_crossentropy', optimizer='adam' )
    return model

batch_size = 16
timesteps = 10
num_actions = 6
model = make_model( num_actions, 1, (84,84,3) )
model.summary()

# Fake training batch. Would be pulled from a replay memory
batch = np.random.uniform( low=0, high=255, size=(batch_size,timesteps,84,84,3) )
y = np.random.randint( 0, high=5, size=(160) )
y = to_categorical( y, num_classes=num_actions )
y = y.reshape( batch_size, timesteps, num_actions )

# Need to find a way to prevent the optimizer from updating every b, but accumulate updates over an entire batch (batch_size).
for b in range(batch_size):
    pred = model.train_on_batch( np.reshape(batch[b,:], (1,timesteps,84,84,3)), np.reshape(y[b,:], (1,timesteps,num_actions)) )
    #for t in range(timesteps):
    #    pred = model.train_on_batch( np.reshape(batch[b,t,:], (1,1,84,84,3)), np.reshape(y[b,t,:], (1,1,num_actions)) )
    model.reset_states() # Don't carry internal state between batches

# move trained network to robot

# This works, but it isn't practical to not get outputs (actions) until after 10 timesteps
#batch = np.random.uniform( low=0, high=255, size=(1,timesteps,84,84,3) )
#pred = model.predict( batch, batch_size=1 )

# This is what I would need to do on my robot, with the LSTM keeping state between calls to predict
max_time = 10 # or 100000, or forever, etc.
for i in range(max_time) :
    image = np.random.uniform( low=0, high=255, size=(1,1,84,84,3) ) # pull one image from camera
    # stateful should be true here
    pred = model.predict( image, batch_size=1 )
    # take action based on pred
    print( pred )

1 个答案:

答案 0 :(得分：1)

您需要的第一件事就是了解您的数据。

这5个维度是否意味着什么？

我会猜测：

- 1 learning example
- 1 time step (this is added by TimeDistributed, normal 2D convolutions don't take this)
- 84 image side
- 84 another image side
- 3 channels (RGB)

TimeDistributed的目的是添加额外的timesteps维，因此您可以模拟不应该使用序列的图层序列。

您的错误消息告诉您：

  • 您的input_shape参数为(None, 10, 84, 84, 3)，其中无是批量大小（样本数/示例数）。
  • 您的输入data，代码batch为(1, 1, 84, 84, 3)。input_shape。

存在不匹配，您应该使用包含10个时间步骤的批次（由stateful=False定义）。 stateful=True模型可以批量打包10个图像，然后用它进行训练。

但稍后，在input_shape案例中，您需要None只需一步。 （您可以创建一个新模型，仅用于预测和复制从训练模型到预测模型的所有权重，或者您可以尝试在该时间步长维度中使用(None, timeSteps, features)，这意味着您可以使用不同的数量训练和预测时间步骤）

现在，与卷积不同，LSTM层已经预期时间步长。因此，您应该找到一种以较小的维度挤压数据的方法。

LSTM预计会None。时间步骤与前一步骤相同。 10用于培训，1用于预测，您可以尝试使用Flatten()。

因此，代替TimeDistributed内的model.add(Reshape((8,9*9*32))) #the batch size doesn't participate in this definition, and it will remain as it is. ，您应该简单地重塑数据，缩小非批量大小或步骤的维度：

9*9*32

model.summary()是前一个卷积的一侧及其32个过滤器。 （我不确定边是9，也许他们是8，你可以在当前的stateful=True中看到）。

最后，对于batch_shape案例，您必须使用input_shape而不是for filename in $1*;do if("$filename" == "hello.txt");then echo "WOW!" fi done 来定义模型。批次中的样品量必须是固定数量，因为模型将假定第二批中的样品是属于上一批样品的新步骤。 （然后所有批次的样本数量必须相同）。

相关问题
最新问题