我有6层CNN培训问题... 我正在处理64x64位的黑白图片... 我有150k样本和大约800类图像 我试图用6层实现CNN ..但是我在训练期间遇到了麻烦...即使在300或400个训练时期内，训练精度总是很低......而且值总是相似的...所以我认为某处应该有一个大错误，但我还没有找到它。 我是一个类数据集（此处没有给出具有3个成员的实现.. dataset.train，dataset.test，dataset.validation和每个成员都有一个方法next_batch sfuffle并正确返回样本（这是我想念的第一件事，我错过了）。我也尝试使用tensorflow的热门编码，但不知道用法是否正确。或者有一个更聪明的。除此之外，由于测试装置不适合我的图形卡存储器，我分批进行测试验证，然后计算准确度。 我也试图减少数据集和类的数量，但结果相同...所以我认为这是一个与我的模型有关的问题...... （网络架构源自This article） 我试图改变完全连接层的神经元数量以及批量大小，但没有运气...... 谁能指出我正确的方向？
这是我得到的输出......
# Image size is also equal to the number of feature...imagine a flat image :)
img_width = 64
img_height = 64
image_size = img_width * img_height
num_classes = 800
# training data and labels
# First dimension (set to None. will be the batch number aka the number of sample used
# at the same time in our training...Setting to None let us choose dynamically
# whatever value)
x = tf.placeholder(tf.float32, shape=[None, image_size])
y_ = tf.placeholder(tf.float32, shape=[None, num_classes])
# Dropout keep probability
keep_prob = tf.placeholder(tf.float32)
# Those will helper function to initialize be our weight and our biases of the network...
def weight_variable(shape):
initial = tf.truncated_normal(shape, stddev=0.1)
return tf.Variable(initial)
def bias_variable(shape):
initial = tf.constant(0.1, shape=shape)
return tf.Variable(initial)
# Convolutional of size x and strides 1 so initial image size is kept
# padding=SAME
def conv2d(x, W):
return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')
# 2 x 2 max pool with stride 2
def max_pool_2x2(x):
return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],
strides=[1, 2, 2, 1], padding='SAME')
######################################
# First layer Convolution
######################################
# 3x3 patch, 1 input channel (the image), 32 output channels
W_conv1 = weight_variable([3, 3, 1, 32])
b_conv1 = bias_variable([32])
# Reshape our image to matrix multiply and number of colors channel (we have B/N images)
x_image = tf.reshape(x, [-1, img_width, img_height , 1])
# multiply and then Relu
h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)
# and then dropout
h_drop1 = tf.nn.dropout(h_conv1, keep_prob)
######################################
# Second layer Convolution M6-1 network
######################################
# 3x3 patch, 32 input channel (the previous output channels), 32 output channels
W_conv2 = weight_variable([3, 3, 32, 32])
b_conv2 = bias_variable([32])
# multiply and then Relu
h_conv2 = tf.nn.relu(conv2d(h_drop1, W_conv2) + b_conv2)
# and the max pool (so image is now 32 x 32)
h_pool2 = max_pool_2x2(h_conv2)
# and then dropout
h_drop2 = tf.nn.dropout(h_pool2, keep_prob)
######################################
# Third layer Convolution M6-1 network
######################################
# 3x3 patch, 32 input channel (the previous output channels), 64 output channels
W_conv3 = weight_variable([3, 3, 32, 64])
b_conv3 = bias_variable([64])
# multiply and then Relu
h_conv3 = tf.nn.relu(conv2d(h_drop2, W_conv3) + b_conv3)
# and then dropout
h_drop3 = tf.nn.dropout(h_conv3, keep_prob)
######################################
# Forth layer Convolution M6-1 network
######################################
# 3x3 patch, 64 input channel (the previous output channels), 64 output channels
W_conv4 = weight_variable([3, 3, 64, 64])
b_conv4 = bias_variable([64])
# multiply and then Relu
h_conv4 = tf.nn.relu(conv2d(h_drop3, W_conv4) + b_conv4)
# and the max pool image is now 16 x 16
h_pool4 = max_pool_2x2(h_conv4)
# and then dropout
h_drop4 = tf.nn.dropout(h_pool4, keep_prob)
######################################
# Fifth layer fully connected M6-1 network
######################################
# Fully connected layer
# Our image is now 16 x 16 after 2 max pools...and we have 64 channels so
# we want a matrix 16 * 16 * 64 and then 256 neurons
W_fc1 = weight_variable([16 * 16 * 64, 256])
b_fc1 = bias_variable([256])
h_pool4_flat = tf.reshape(h_drop4, [-1, 16 * 16 * 64])
h_fc1 = tf.nn.relu(tf.matmul(h_pool4_flat, W_fc1) + b_fc1)
######################################
# Output layer fully connected M6-1 network
######################################
# Fully connected layer
# Our image is now 16 x 16 after 2 max pools...and we have 256 channels so
# we want a matrix 16 * 16 * 64 and then 256 neurons
W_fc2 = weight_variable([256, num_classes])
b_fc2 = bias_variable([num_classes])
# Softmax classifier
y_predict = tf.nn.softmax(tf.matmul(h_fc1, W_fc2) + b_fc2)
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y_predict), reduction_indices=[1]))
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)
correct_prediction = tf.equal(tf.argmax(y_predict,1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
sess = tf.Session()
sess.run(tf.initialize_all_variables())
batch_size = 128
max_train_epoch = 1000
while dataset.train.epochs_completed != max_train_epoch:
batch = dataset.train.next_batch(batch_size)
y_hot_one = tf.contrib.layers.one_hot_encoding(labels=batch[1], num_classes=num_classes)
if dataset.train.index_in_epoch % 10 * batch_size == 0:
train_accuracy = accuracy.eval(session=sess, \
feed_dict={x:batch[0], y_: y_hot_one.eval(session=sess), keep_prob: 1.0})
print("epoch: %d step %d, training accuracy %g" \
% (dataset.train.epochs_completed, dataset.train.index_in_epoch, train_accuracy))
train_step.run(session=sess, \
feed_dict={x: batch[0], y_: y_hot_one.eval(session=sess), keep_prob: 0.5})
start = 0
size = 1000
if dataset.test.num_examples < size:
end = dataset.test.num_examples
else:
end = size
count = 1
total = []
print "Total test Example ", dataset.test.num_examples
while end <= dataset.test.num_examples:
print "Start index ", start, " end ", end
test_label_hot_one = tf.contrib.layers.one_hot_encoding(labels=dataset.test.labels[start:end], num_classes=num_classes)
correct = correct_prediction.eval(session=sess, \
feed_dict={x: dataset.test.data[start:end], y_: test_label_hot_one.eval(session=sess), keep_prob: 1.0})
start = end
end += size
if end > dataset.test.num_examples:
end = dataset.test.num_examples
total.extend(np.ravel(correct))
#print "total = ", total
total_corrected = sum(total)
#print type(summa)
percentage = (total_corrected / float(dataset.test.num_examples)) * 100
print("Test accuracy %g %d/%d" % (percentage, total_corrected, dataset.test.num_examples))
当我运行代码时......
> BlockquoteI tensorflow/core/common_runtime/gpu/gpu_init.cc:126] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_init.cc:136] 0: Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:839] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 960M, pci bus id: 0000:01:00.0)
epoch: 0 step 640, training accuracy 0.015625
epoch: 0 step 1280, training accuracy 0
epoch: 0 step 1920, training accuracy 0
epoch: 0 step 2560, training accuracy 0
epoch: 0 step 3200, training accuracy 0
epoch: 0 step 3840, training accuracy 0
epoch: 0 step 4480, training accuracy 0
epoch: 0 step 5120, training accuracy 0
epoch: 0 step 5760, training accuracy 0
epoch: 0 step 6400, training accuracy 0
epoch: 0 step 7040, training accuracy 0
epoch: 0 step 7680, training accuracy 0
epoch: 0 step 8320, training accuracy 0
epoch: 0 step 8960, training accuracy 0
epoch: 0 step 9600, training accuracy 0
epoch: 0 step 10240, training accuracy 0
epoch: 0 step 10880, training accuracy 0
epoch: 0 step 11520, training accuracy 0.0078125
epoch: 0 step 12160, training accuracy 0
epoch: 0 step 12800, training accuracy 0
epoch: 0 step 13440, training accuracy 0
epoch: 0 step 14080, training accuracy 0.0078125
如果我减小数据集的大小（12k样本）以及标签数量75 我得到了...
I tensorflow/core/common_runtime/gpu/gpu_init.cc:126] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_init.cc:136] 0: Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:839] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 960M, pci bus id: 0000:01:00.0)
epoch: 0 step 640, training accuracy 0.0078125
epoch: 0 step 1280, training accuracy 0.015625
epoch: 0 step 1920, training accuracy 0.015625
epoch: 0 step 2560, training accuracy 0.0234375
epoch: 0 step 3200, training accuracy 0.0234375
epoch: 0 step 3840, training accuracy 0.03125
epoch: 0 step 4480, training accuracy 0.0234375
epoch: 0 step 5120, training accuracy 0.0078125
epoch: 0 step 5760, training accuracy 0.0078125
epoch: 0 step 6400, training accuracy 0.0078125
epoch: 0 step 7040, training accuracy 0.0234375
Epochs completed 1
epoch: 1 step 640, training accuracy 0
epoch: 1 step 1280, training accuracy 0.015625
epoch: 1 step 1920, training accuracy 0.0234375
epoch: 1 step 2560, training accuracy 0
epoch: 1 step 3200, training accuracy 0.015625
epoch: 1 step 3840, training accuracy 0.015625
epoch: 1 step 4480, training accuracy 0.015625
epoch: 1 step 5120, training accuracy 0.03125
epoch: 1 step 5760, training accuracy 0.0234375
epoch: 1 step 6400, training accuracy 0.0234375
epoch: 1 step 7040, training accuracy 0.0234375
即使在500个训练时期之后，这个数字仍然几乎相同。
任何人都能指出我正确的方向吗？ 问候。