TensorFlow：为预先训练的重量添加一点噪音

Question

我已经完成了3个LSTM堆栈的训练，现在我想研究每个LSTM层对其权重的小扰动有多敏感。我想从检查点加载模型，为某个层的权重添加一个小值并记录性能损失/收益。我想知道最简单的方法是什么？

Answer 1

import tensorflow as tf

# define model
...

# load checkpoint
...

# assemble the list of weights to add noise
list_of_weights = [ ... ] 


def add_random_noise(w, mean=0.0, stddev=1.0):
    variables_shape = tf.shape(w)
    noise = tf.random_normal(
        variables_shape,
        mean=mean,
        stddev=stddev,
        dtype=tf.float32,
    )
    return tf.assign_add(w, noise)


sess = tf.Session()
for w in list_of_weights:
    sess.run(add_random_noise(w))

# continue experiments
...