Question

我构建了一个LSTM网络来对序列数据进行回归。当我尝试激活隐藏层（即LSTM层）时，它返回零。网络只有一个隐藏层，一个输入层和一个输出层。

我尝试使用以下代码段获取隐藏的图层值。

print net.activate(data)
print net['in'].outputbuffer[net['in'].offset]
print net['hidden0'].outputbuffer[net['hidden0'].offset]

知道为什么吗？以下是更完整的代码段

RopewayIn = RopewayOverallData[:-1, :]
RopewayOut = RopewayOverallData[1:, :]
ds.newSequence()
for i in range(noDataFrames):
    ds.appendLinked( [RopewayIn[i,0],RopewayIn[i,1], RopewayIn[i,2], RopewayIn[i,3], RopewayIn[i,4], RopewayIn[i,5], RopewayIn[i,6], RopewayIn[i,7], RopewayIn[i,8], RopewayIn[i,9]], 
                     [RopewayOut[i,0],RopewayOut[i,1], RopewayOut[i,2], RopewayOut[i,3], RopewayOut[i,4], RopewayOut[i,5], RopewayOut[i,6], RopewayOut[i,7], RopewayOut[i,8], RopewayOut[i,9]] )

net = buildNetwork(10,20,10, hiddenclass=LSTMLayer,outclass=LinearLayer, bias=True, recurrent=True) 
trainer = RPropMinusTrainer(net, dataset=ds, verbose=True, weightdecay=0.01)

for i in range(10001):
     trainer.trainEpochs(2)

print net.activate(RopewayOverallData[0,4])
print net['in'].outputbuffer[net['in'].offset]
print net['hidden0'].outputbuffer[net['hidden0'].offset

Answer 1

这不是一个真正的答案，但它不适合评论。我尝试运行此代码，混合使用您的代码和上一个问题（activation values for all nodes in a PyBrain network）：

from pybrain.tools.shortcuts import buildNetwork
from pybrain.datasets import SupervisedDataSet
from pybrain.supervised.trainers import BackpropTrainer
from pybrain.structure.modules import LSTMLayer,LinearLayer

net = buildNetwork(3,3,3, hiddenclass=LSTMLayer,outclass=LinearLayer, bias=True, recurrent=True)
dataSet = SupervisedDataSet(3, 3)
dataSet.addSample((0, 0, 0), (0, 0, 0))
dataSet.addSample((1, 1, 1), (0, 0, 0))
dataSet.addSample((1, 0, 0), (1, 0, 0))
dataSet.addSample((0, 1, 0), (0, 1, 0))
dataSet.addSample((0, 0, 1), (0, 0, 1))

trainer = BackpropTrainer(net, dataSet)
trained = False
acceptableError = 0.001

howmanytries = 0
# train until acceptable error reached
while (trained == False) and (howmanytries < 1000):
    error = trainer.train()
    if error < acceptableError :
        trained = True
    else:
        howmanytries += 1

result = net.activate([0.5, 0.4, 0.7])
net['in'].outputbuffer[net['in'].offset]
net['hidden0'].outputbuffer[net['hidden0'].offset]
print result

...它打印出非常好的非零结果。我将从这开始，将一块一块地改回到你的代码中，看看它在哪里停止工作。

pybrain LSTM层的激活为零

1 个答案: