Question

阅读Tensorflow rnn cells source code我遇到了以下GRUCell代码：

self._gate_kernel = self.add_variable(
    "gates/%s" % _WEIGHTS_VARIABLE_NAME,
    shape=[input_depth + self._num_units, 2 * self._num_units],
    initializer=self._kernel_initializer)
self._gate_bias = self.add_variable(
    "gates/%s" % _BIAS_VARIABLE_NAME,
    shape=[2 * self._num_units],
    initializer=(
        self._bias_initializer
        if self._bias_initializer is not None
        else init_ops.constant_initializer(1.0, dtype=self.dtype)))
self._candidate_kernel = self.add_variable(
    "candidate/%s" % _WEIGHTS_VARIABLE_NAME,
    shape=[input_depth + self._num_units, self._num_units],
    initializer=self._kernel_initializer)
self._candidate_bias = self.add_variable(
    "candidate/%s" % _BIAS_VARIABLE_NAME,
    shape=[self._num_units],
    initializer=(
        self._bias_initializer
        if self._bias_initializer is not None
        else init_ops.zeros_initializer(dtype=self.dtype)))

你可以看到门偏置张量用1和候选偏置用0初始化，而门内核和候选内核张量没有默认初始化器（self._kernel_initializer=None）如果我们不'明确提供它们。那么Tensorflow是否用随机数填充这些张量？

Answer 1

此处的Function FindSmallest(arr) out = arr(0) For i = 1 to UBound(arr) If out > arr(i) Then out = arr(i) End If Next FindSmallest = out End Function WScript.Echo FindSmallest(Array(var1,var2,var3,var4))方法最终会调用add_variable here。

如您所见，如果未指定初始化程序且dtype是浮点数，则使用add_weight。

Tensroflow如何在LSTM / GRU单元中初始化内核权重？

1 个答案: