Question

I'm building a simple image segmentation network, based off the encoder/decoder architecture in TensorFlow and currently the model seems to run but it's not updating the weights. The inputs are my own dataset: images converted to numpy arrays of the shape [-1,200,200,1] with pixel-bu-pixel mask labels of shape [-1,40000,3].

The code for my model function is here

def model(features, labels, mode):

    inpt = tf.reshape(features["x"], [-1,200,200,1])

    #encoder
    x = tf.layers.conv2d(
        inputs = inpt,
        filters = 16,
        kernel_size = [2,2],
        padding = 'same',
        activation = tf.nn.relu)  
    x = tf.layers.max_pooling2d(inputs = x, pool_size = [2,2], strides = 2)

    x = tf.layers.conv2d(
        inputs = x,
        filters = 32,
        kernel_size = [2,2],
        padding = 'same',
        activation = tf.nn.relu)
    x = tf.layers.max_pooling2d(inputs = x, pool_size = [2,2], strides = 2)

    x = tf.layers.conv2d(
        inputs = x,
        filters = 64,
        kernel_size = [2,2],
        padding = 'same',
        activation = tf.nn.relu)
    x = tf.layers.max_pooling2d(inputs = x, pool_size = [2,2], strides = 2)

    #decoder
    x = tf.keras.layers.UpSampling2D(size = (2,2))(x)
    x = tf.layers.conv2d(
        inputs = x,
        filters = 64,
        kernel_size = [2,2],
        padding = 'same',
        activation = tf.nn.relu)

    x = tf.keras.layers.UpSampling2D(size = (2,2))(x)
    x = tf.layers.conv2d(
        inputs = x,
        filters = 32,
        kernel_size = [2,2],
        padding = 'same',
        activation = tf.nn.relu)

    x = tf.keras.layers.UpSampling2D(size = (2,2))(x)   
    x = tf.layers.conv2d(
        inputs = x,
        filters = 16,
        kernel_size = [2,2],
        padding = 'same',
        activation = tf.nn.relu)

    x = tf.layers.conv2d(
        inputs = x,
        filters = 3,
        kernel_size = [1,1],
        padding = 'valid',
        activation = tf.nn.relu)

    logits = tf.reshape(x, [-1,3])

    predictions = {
        "classes": tf.argmax(input = logits, axis = 1),
        "probabilities": tf.nn.softmax(logits, name = "softmax_tensor")
    }

    if mode == tf.estimator.ModeKeys.PREDICT:
        return tf.estimator.EstimatorSpec(mode = mode, predictions = predictions)

    labels = tf.reshape(labels, [-1,3])

    loss = tf.losses.softmax_cross_entropy(onehot_labels = labels, logits = logits)

    if mode == tf.estimator.ModeKeys.TRAIN:
        optimizer = tf.train.GradientDescentOptimizer(learning_rate = 0.0001)
        train_op = optimizer.minimize(
            loss = loss,
            global_step = tf.train.get_global_step())
        return tf.estimator.EstimatorSpec(mode = mode, loss = loss, train_op = train_op)


    eval_metric_ops = {
        "accuracy": tf.metrics.accuracy(
            labels = tf.argmax(labels, axis = 1), predictions = predictions["classes"])
    }
    return tf.estimator.EstimatorSpec(
        mode = mode, loss = loss, eval_metric_ops = eval_metric_ops)

I then call the model as follows:

classifier = tf.estimator.Estimator(model_fn = model, model_dir = '.')

tensors_to_log = {"probabilities": "softmax_tensor"}
logging_hook = tf.train.LoggingTensorHook(
    tensors = tensors_to_log, every_n_iter = 50)

train_input_fn = tf.estimator.inputs.numpy_input_fn(
    x = {"x": train},
    y = train_label,
    batch_size = 1,
    num_epochs = 5,
    shuffle = True)

classifier.train(
    input_fn = train_input_fn,
    steps = None)

The model is running but the accuracy score remains around the 35% mark and fails to train. If anyone has any suggestions they would be most appreciated.

Answer 1

在问题的开头你说：

模型似乎在运行，但它没有更新权重。

然后，最后你说：

准确度得分仍然在35％左右，未能训练

这是两种不同的情况。在第一种情况下，权重根本不更新，而在第二种情况下，权重被更新，但是它们的更新在准确性方面没有任何改进。您首先需要检查您的情况。

恒定重量

如果在训练期间权重保持不变，您可能正在试验gradient vanishing。你的网络已经作为激活功能，所以可能几乎所有的线性输出（也就是说Z = W*x + b）都是负的，而relu的梯度等于0表示负输入，所以学习不会通过层反向传播到每个重量。您可以尝试不同的激活功能，或者在CNN中更常见的是在每个转换层之后添加批量标准化层。

恒定准确度

你有几个原因。

您的网络可能处于全局最优状态，因此无论您做什么都没关系，如果您不扩展数据集，则无法达到更高的准确度
您的网络可能处于本地最佳状态。即使由于权重的随机初始化而不太可能，您应该尝试在每次迭代时实施minibatch和/或随机播放数据集。
您的网络很简单，无法理解输入和输出之间的基础关系。
您的网络过于复杂。在这种情况下，如果您为大量时期留下网络培训，则应尝试过度拟合。

希望它有所帮助！

Tensorflow Image Segmentation weights not updating

1 个答案:

恒定重量

恒定准确度