我想检查在神经网络训练期间是否真正添加和使用了噪声。因此,我用这样的keras建立我的NN:
from keras.layers import Input
from keras.layers.noise import GaussianNoise
inp = Input(tensor=self.X_ph)
noised_x = GaussianNoise(stddev=self.x_noise_std)(inp)
x = Dense(15, activation='elu')(noised_x)
x = Dense(15, activation='elu')(x)
self.estimator = x
...
# kernel weights, as output by the neural network
self.logits = logits = Dense(n_locs * self.n_scales, activation='softplus')(self.estimator)
self.weights = tf.nn.softmax(logits)
# mixture distributions
self.cat = cat = Categorical(logits=logits)
self.components = components = [MultivariateNormalDiag(loc=loc, scale_diag=scale) for loc in locs_array for scale in scales_array]
self.mixtures = mixtures = Mixture(cat=cat, components=components, value=tf.zeros_like(self.y_ph))
然后我使用edward来执行推理:
self.inference = ed.MAP(data={self.mixtures: self.y_ph})
self.inference.initialize(var_list=tf.trainable_variables(), n_iter=self.n_training_epochs)
tf.global_variables_initializer().run()
根据文档,我最接近的是通过ed.MAP的run()
和update()
函数。
最好,我会做这样的事情:
noised_x = self.sess.run(self.X_ph, feed_dict={self.X_ph: X, self.y_ph: Y})
np.allclose(noised_x, X) --> False
显然我使用GaussianNoise的方式似乎没有给我的输入增加噪音,因为以下单位测试失败了:
X, Y = self.get_samples(std=1.0)
model_no_noise = KernelMixtureNetwork(n_centers=5, x_noise_std=None, y_noise_std=None)
model_no_noise.fit(X,Y)
var_no_noise = model_no_noise.covariance(x_cond=np.array([[2]]))[0][0][0]
model_noise = KernelMixtureNetwork(n_centers=5, x_noise_std=20.0, y_noise_std=20.0)
model_noise.fit(X, Y)
var_noise = model_noise.covariance(x_cond=np.array([[2]]))[0][0][0]
self.assertGreaterEqual(var_noise - var_no_noise, 0.1)
我还确保在inference.update(...)
断言期间
assert tf.keras.backend.learning_phase() == 1
通行证。
这里可能出现问题?