Tensorflow模型在推理期间输出不同的值

Question

在训练期间，我记录了我的回归模型输出的预测值与训练数据。当我在预测模式下运行相同的数据集时，模型输出的值范围会有所不同：

Tensorboard

在这里，我们看到模型已经预测了范围内的值（140,250）。

针对同一数据集的预测

这里我们有模型预测值（17,23）。是什么给了什么？

我怀疑使用moving_mean时，估算工具API并没有神奇地保存moving_variance和tf.layers.batch_normalization。

我的模特：

def model_fn(features, labels, mode, params):
  training = mode == tf.estimator.ModeKeys.TRAIN
  extra_update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)

  x = tf.reshape(features, [-1, 32, 32, 3])
  x = tf.layers.batch_normalization(x, training=training, name='norm_128')

  i = 1
  for filters in [32, 64]:
    x = tf.layers.conv2d(x, filters=filters, kernel_size=3, activation=None, name='conv{}'.format(i))
    x = tf.layers.batch_normalization(x, training=training, name='norm{}'.format(i))
    x = tf.nn.relu(x, name='act{}'.format(i))
    i += 1

    x = tf.layers.conv2d(x, filters=filters * 2, kernel_size=3, strides=2, activation=None, name='pool{}'.format(i))
    x = tf.layers.batch_normalization(x, training=training, name='norm{}'.format(i))
    x = tf.nn.relu(x, name='act{}'.format(i))
    i += 1

  flat = tf.contrib.layers.flatten(x, scope='flatten')
  dropout = tf.layers.dropout(flat, rate=params['dropout_rate'], training=training, name='dropout')
  output_layer = tf.layers.dense(dropout, units=1, name='output_layer')

  predictions = tf.reshape(output_layer, [-1])

  predictions_dict = {
    'pred': predictions,
  }

  if mode == tf.estimator.ModeKeys.PREDICT:
    return tf.estimator.EstimatorSpec(mode=mode, predictions=predictions_dict)

  loss = tf.losses.mean_squared_error(labels=labels, predictions=predictions)

  tf.summary.scalar('loss', loss)
  tf.summary.histogram('prediction', predictions)
  tf.summary.scalar('prediction', tf.reduce_mean(predictions))

  optimizer = tf.train.AdamOptimizer(learning_rate=params['learning_rate'])
  with tf.control_dependencies(extra_update_ops):
    train_op = optimizer.minimize(loss=loss, global_step=tf.train.get_global_step())
  eval_metric_ops = {
    'rmse_val': tf.metrics.root_mean_squared_error(labels=tf.cast(labels, tf.float32), predictions=predictions)
  }

  tf.summary.scalar('rmse_train', eval_metric_ops['rmse_val'][1])


  return tf.estimator.EstimatorSpec(mode=mode, loss=loss, train_op=train_op, eval_metric_ops=eval_metric_ops)

Answer 1

<强> __编辑__

代码中唯一的随机点是drop out。在列车之后和预测时间，将保持丢失的概率设置为1。因为辍学层选择随机传递给它们的变量子集，并且在该子集上进行训练以防止过度拟合。