Question

我正在尝试制作一个简单的MLP来预测图像像素的值 - original blog。这是我之前在python中使用Keras的尝试 - link

我尝试在tensorflow中做同样的事情，但是当它们应该小于1时，我得到非常大的输出值（~10 ^ 12）。

这是我的代码：

import numpy as np
import cv2
from random import shuffle
import tensorflow as tf

'''
Image preprocessing
'''
image_file = cv2.imread("Mona Lisa.jpg")

h = image_file.shape[0]
w = image_file.shape[1]

preX = []
preY = []

for i in xrange(h):
    for j in xrange(w):
        preX.append([i,j])
        preY.append(image_file[i,j,:].astype('float32')/255.0)

print preX[:5], preY[:5]
zipped = [i for i in zip(preX,preY)]
shuffle(zipped)

X_train = np.array([i for (i,j) in zipped]).astype('float32')
Y_train = np.array([j for (i,j) in zipped]).astype('float32')

print X_train[:10], Y_train[:10]

'''
Tensorflow code
'''

def weight_variable(shape):
  initial = tf.truncated_normal(shape, stddev=0.1)
  return tf.Variable(initial)

def bias_variable(shape):
  initial = tf.constant(0.1, shape=shape)
  return tf.Variable(initial)

x = tf.placeholder(tf.float32, shape=[None,2])
y = tf.placeholder(tf.float32, shape=[None,3])



'''
Layers
'''

w1 = weight_variable([2,300])
b1 = bias_variable([300])
L1 = tf.nn.relu(tf.matmul(X_train,w1)+b1)

w2 = weight_variable([300,3])
b2 = bias_variable([3])
y_model = tf.matmul(L1,w2)+b2


'''
Training
'''

# criterion
MSE = tf.reduce_mean(tf.square(tf.sub(y,y_model)))

# trainer
train_op = tf.train.GradientDescentOptimizer(learning_rate = 0.01).minimize(MSE)

nb_epochs = 10

init = tf.initialize_all_variables()
sess = tf.Session()

sess.run(init)
cost = 0

for i in range(nb_epochs):
    sess.run(train_op, feed_dict ={x: X_train, y: Y_train})
    cost += sess.run(MSE, feed_dict ={x: X_train, y: Y_train})

cost /= nb_epochs
print cost


'''
Prediction
'''

pred = sess.run(y_model,feed_dict = {x:X_train})*255.0
print pred[:10]

output_image = []
index = 0

h = image_file.shape[0]
w = image_file.shape[1]

for i in xrange(h):
    row = []

    for j in xrange(w):
        row.append(pred[index])
        index += 1

    row = np.array(row)
    output_image.append(row)

output_image = np.array(output_image)
output_image = output_image.astype('uint8')
cv2.imwrite('out_mona_300x3_tf.png',output_image)

Answer 1

首先，我认为不是运行train_op而是运行MSE 您可以在列表中运行两个操作并显着降低计算成本。

for i in range(nb_epochs):
cost += sess.run([MSE, train_op], feed_dict ={x: X_train, y: Y_train})

其次，我建议总是写出你的成本函数，这样你就可以看到在训练阶段发生了什么。手动将其打印出来或使用张量板记录您的成本并绘制它（您可以在官方tf页面上找到示例）。您还可以监控您的体重，看它们是不是在爆炸。

您可以尝试一些事项：降低学习率，为权重添加正则化。检查您的训练集（像素）是否真的包含值你期待他们。

Answer 2

您为输入图层权重和输出图层赋予相同的名称w和b，因此在渐变 - 下降过程中似乎出现了问题。实际上我很惊讶张量流不会发出错误或者发出警告（或者我错过了什么？）

在线性回归中获得非常高的值

2 个答案: