Question

keras.layers.GRU(units, activation='tanh', recurrent_activation='hard_sigmoid', use_bias=True, kernel_initializer='glorot_uniform', recurrent_initializer='orthogonal', bias_initializer='zeros', kernel_regularizer=None, recurrent_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, recurrent_constraint=None, bias_constraint=None, dropout=0.0, recurrent_dropout=0.0, implementation=1, return_sequences=False, return_state=False, go_backwards=False, stateful=False, unroll=False)

我知道输入权重的初始值是＆＃39; glorot_uniform＆＃39;并且隐藏统计数据的权重的初始化程序是正交的＆＃39;而偏见的初始化者是“零”。

但我不知道GRU的第一个隐藏状态的初始化程序

Answer 1

它是零。您可以在recurrent.py中找到以下行。

def get_initial_state(self, inputs):
    # build an all-zero tensor of shape (samples, output_dim)
    initial_state = K.zeros_like(inputs)  # (samples, timesteps, input_dim)
    initial_state = K.sum(initial_state, axis=(1, 2))  # (samples,)
    initial_state = K.expand_dims(initial_state)  # (samples, 1)
    if hasattr(self.cell.state_size, '__len__'):
        return [K.tile(initial_state, [1, dim])
                for dim in self.cell.state_size]
    else:
        return [K.tile(initial_state, [1, self.cell.state_size])]

当您调用GRU图层而未指定初始状态且图层不具有状态时，此函数用于获取第一个隐藏状态，即常用用法GRU(...)(x)。

def call(self,
         inputs,
         mask=None,
         training=None,
         initial_state=None,
         constants=None):
    # input shape: `(samples, time (padded with zeros), input_dim)`
    # note that the .build() method of subclasses MUST define
    # self.input_spec and self.state_spec with complete input shapes.
    if isinstance(inputs, list):
        inputs = inputs[0]
    if initial_state is not None:
        pass
    elif self.stateful:
        initial_state = self.states
    else:
        initial_state = self.get_initial_state(inputs)

KERAS中GRU的默认隐藏状态是什么？

1 个答案: