Question

我正在尝试实施“Livelinet：多模式深度回归神经网络预测教育视频中的活力”的结构。

为了简要说明，我将10秒音频片段分成10个1秒音频片段，并从该1秒音频片段中获取谱图（图片）。然后我使用CNN从图片中获取一个表示向量，最后得到每个1秒视频剪辑的10个向量。

接下来，我将这10个向量提供给LSTM，我在那里遇到了一些错误。我的代码和错误回溯如下：

class AudioCNN(nn.Module):

def __init__(self):
    super(AudioCNN,self).__init__()
    self.features = alexnet.features
    self.features2 = nn.Sequential(*classifier)
    self.lstm = nn.LSTM(512, 256,2)
    self.classifier = nn.Linear(2*256,2)

def forward(self, x):
    x = self.features(x)
    print x.size()
    x = x.view(x.size(0),256*6*6)
    x = self.features2(x)
    x = x.view(10,1,512)
    h_0,c_0 = self.init_hidden()
    _, (_, _) = self.lstm(x,(h_0,c_0)) # x dim : 2 x 1 x 256
    assert False
    x = x.view(1,1,2*256)
    x = self.classifier(x)

    return x

def init_hidden(self):
    h_0 = torch.randn(2,1,256) #layer * batch * input_dim
    c_0 = torch.randn(2,1,256)
    return h_0, c_0

audiocnn = AudioCNN()
input = torch.randn(10,3,223,223)
input = Variable(input)
audiocnn(input)

错误：

RuntimeErrorTraceback (most recent call last)
<ipython-input-64-2913316dbb34> in <module>()
----> 1 audiocnn(input)

/home//local/lib/python2.7/site-packages/torch/nn/modules/module.pyc in __call__(self, *input, **kwargs)
    222         for hook in self._forward_pre_hooks.values():
    223             hook(self, input)
--> 224         result = self.forward(*input, **kwargs)
    225         for hook in self._forward_hooks.values():
    226             hook_result = hook(self, input, result)

<ipython-input-60-31881982cca9> in forward(self, x)
     15         x = x.view(10,1,512)
     16         h_0,c_0 = self.init_hidden()
---> 17         _, (_, _) = self.lstm(x,(h_0,c_0)) # x dim : 2 x 1 x 256
     18         assert False
     19         x = x.view(1,1,2*256)

/home/local/lib/python2.7/site-packages/torch/nn/modules/module.pyc in __call__(self, *input, **kwargs)
    222         for hook in self._forward_pre_hooks.values():
    223             hook(self, input)
--> 224         result = self.forward(*input, **kwargs)
    225         for hook in self._forward_hooks.values():
    226             hook_result = hook(self, input, result)

/home//local/lib/python2.7/site-packages/torch/nn/modules/rnn.pyc in forward(self, input, hx)
    160             flat_weight=flat_weight
    161         )
--> 162         output, hidden = func(input, self.all_weights, hx)
    163         if is_packed:
    164             output = PackedSequence(output, batch_sizes)

/home//local/lib/python2.7/site-packages/torch/nn/_functions/rnn.pyc in forward(input, *fargs, **fkwargs)
    349         else:
    350             func = AutogradRNN(*args, **kwargs)
--> 351         return func(input, *fargs, **fkwargs)
    352 
    353     return forward

/home//local/lib/python2.7/site-packages/torch/nn/_functions/rnn.pyc in forward(input, weight, hidden)
    242             input = input.transpose(0, 1)
    243 
--> 244         nexth, output = func(input, hidden, weight)
    245 
    246         if batch_first and batch_sizes is None:

/home//local/lib/python2.7/site-packages/torch/nn/_functions/rnn.pyc in forward(input, hidden, weight)
     82                 l = i * num_directions + j
     83 
---> 84                 hy, output = inner(input, hidden[l], weight[l])
     85                 next_hidden.append(hy)
     86                 all_output.append(output)

/home//local/lib/python2.7/site-packages/torch/nn/_functions/rnn.pyc in forward(input, hidden, weight)
    111         steps = range(input.size(0) - 1, -1, -1) if reverse else range(input.size(0))
    112         for i in steps:
--> 113             hidden = inner(input[i], hidden, *weight)
    114             # hack to handle LSTM
    115             output.append(hidden[0] if isinstance(hidden, tuple) else hidden)

/home//local/lib/python2.7/site-packages/torch/nn/_functions/rnn.pyc in LSTMCell(input, hidden, w_ih, w_hh, b_ih, b_hh)
     29 
     30     hx, cx = hidden
---> 31     gates = F.linear(input, w_ih, b_ih) + F.linear(hx, w_hh, b_hh)
     32 
     33     ingate, forgetgate, cellgate, outgate = gates.chunk(4, 1)

/home//local/lib/python2.7/site-packages/torch/nn/functional.pyc in linear(input, weight, bias)
    551     if input.dim() == 2 and bias is not None:
    552         # fused op is marginally faster
--> 553         return torch.addmm(bias, input, weight.t())
    554 
    555     output = input.matmul(weight.t())

/home//local/lib/python2.7/site-packages/torch/autograd/variable.pyc in addmm(cls, *args)
    922         @classmethod
    923         def addmm(cls, *args):
--> 924             return cls._blas(Addmm, args, False)
    925 
    926         @classmethod

/home//local/lib/python2.7/site-packages/torch/autograd/variable.pyc in _blas(cls, args, inplace)
    918             else:
    919                 tensors = args
--> 920             return cls.apply(*(tensors + (alpha, beta, inplace)))
    921 
    922         @classmethod

RuntimeError: save_for_backward can only save input or output tensors, but argument 0 doesn't satisfy this condition

Answer 1

错误消息

RuntimeError: save_for_backward can only save input or output tensors, but argument 0 doesn't satisfy this condition

通常表示您正在传递张量或其他无法将历史记录存储为模块的输入。在您的情况下，您的问题是您在init_hidden()而不是Variable实例中返回张量。因此，当LSTM运行时，它无法计算隐藏层的梯度，因为其初始输入不是backprop图的一部分。

解决方案：

def init_hidden(self):
    h_0 = torch.randn(2,1,256) #layer * batch * input_dim
    c_0 = torch.randn(2,1,256)
    return Variable(h_0), Variable(c_0)

平均值0和方差1也可能无助于LSTM隐藏状态的初始值。理想情况下，您也可以使初始状态也可训练，例如：

h_0 = torch.zeros(2,1,256) # layer * batch * input_dim
c_0 = torch.zeros(2,1,256)
h_0_param = torch.nn.Parameter(h_0)
c_0_param = torch.nn.Parameter(c_0)

def init_hidden(self):
    return h_0_param, c_0_param

在这种情况下，网络可以了解最适合的初始状态。请注意，在这种情况下，无需在h_0_param中包含Variable，因为Parameter基本上是Variable require_grad=True。

torch.nn.LSTM运行时错误

1 个答案: