Question

我使用TensorFlow LSTM作为语言模型（我有一系列单词并希望预测下一个单词），并且当我运行语言模型时，我想打印出来每一步的遗忘，输入，变换和输出门的值。我该怎么做？

通过检查https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/rnn/python/ops/rnn_cell.py中的代码，我发现LayerNormBasicLSTMCell类有一个call方法，其中包含我要打印的i, j, f, o个变量。

  def call(self, inputs, state):
    """LSTM cell with layer normalization and recurrent dropout."""
    c, h = state
    args = array_ops.concat([inputs, h], 1)
    concat = self._linear(args)

    i, j, f, o = array_ops.split(value=concat, num_or_size_splits=4, axis=1)
    if self._layer_norm:
      i = self._norm(i, "input")
      j = self._norm(j, "transform")
      f = self._norm(f, "forget")
      o = self._norm(o, "output")

    g = self._activation(j)
    if (not isinstance(self._keep_prob, float)) or self._keep_prob < 1:
      g = nn_ops.dropout(g, self._keep_prob, seed=self._seed)

    new_c = (c * math_ops.sigmoid(f + self._forget_bias)
             + math_ops.sigmoid(i) * g)
    if self._layer_norm:
      new_c = self._norm(new_c, "state")
    new_h = self._activation(new_c) * math_ops.sigmoid(o)

    new_state = core_rnn_cell.LSTMStateTuple(new_c, new_h)
    return new_h, new_state

但是，有一种简单的方法可以将这些变量打印出来吗？或者我必须在我的脚本中基本上重新创建相关的代码行，我在运行LTSM？

Answer 1

我曾经在git问题中提出过类似的问题。响应是原始单元格仅返回c和h（这也是每个步骤的输出y）。如果你想获得内部变量，你需要自己做。

以下是链接：https://github.com/tensorflow/tensorflow/issues/5731

Answer 2

基本上你可以这样做：

首先返回您需要的状态，例如return new_h, new_state, i, j, f, o。要进行此类更改，您应该从TensorFlow复制源代码文件，并将其作为您自己的代码导入到您的代码中。
然后在您的代码中，在session.run(to_return, feed_dict)中，将to_return改为：

output, state, i, j, f, o = lstm_cell(input, state)   
to_return = {
    "new_h": output, 
    "new_state": state,
    "i": i,
    "j": j,
    "f": f, 
    "o": o,
}

results = session.run(to_return, feed_dict) # get what you want from the 
# graph(which are tensors), resulting in results of a dictionary with values
# being numpy arrays. 

print results["i"] # you'll get a numpy array representing the i gate

如何在TensorFlow中打印出LSTM门的值？

2 个答案: