以下代码来自https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/learn/resnet.py的Tensorflow Resnet示例:
# Create the bottleneck groups, each of which contains `num_blocks`
# bottleneck groups.
for group_i, group in enumerate(groups):
for block_i in range(group.num_blocks):
name = 'group_%d/block_%d' % (group_i, block_i)
# 1x1 convolution responsible for reducing dimension
with tf.variable_scope(name + '/conv_in'):
conv = tf.layers.conv2d(
net,
filters=group.num_filters,
kernel_size=1,
padding='valid',
activation=tf.nn.relu)
conv = tf.layers.batch_normalization(conv, training=training)
with tf.variable_scope(name + '/conv_bottleneck'):
conv = tf.layers.conv2d(
conv,
filters=group.bottleneck_size,
kernel_size=3,
padding='same',
activation=tf.nn.relu)
conv = tf.layers.batch_normalization(conv, training=training)
# 1x1 convolution responsible for restoring dimension
with tf.variable_scope(name + '/conv_out'):
input_dim = net.get_shape()[-1].value
conv = tf.layers.conv2d(
conv,
filters=input_dim,
kernel_size=1,
padding='valid',
activation=tf.nn.relu)
conv = tf.layers.batch_normalization(conv, training=training)
# shortcut connections that turn the network into its counterpart
# residual function (identity shortcut)
net = conv + net
这段代码针对每个块运行,具有给定的输出和瓶颈尺寸。这些块定义为:
# Configurations for each bottleneck group.
BottleneckGroup = namedtuple('BottleneckGroup',
['num_blocks', 'num_filters', 'bottleneck_size'])
groups = [
BottleneckGroup(3, 128, 32), BottleneckGroup(3, 256, 64),
BottleneckGroup(3, 512, 128), BottleneckGroup(3, 1024, 256)
]
在通常的做法中,据我所知,这些所谓的ResNets瓶颈层首先用1x1内核减少输入通道数,在减少的通道大小中应用更高阶(3x3)的卷积然后恢复到输入通道大小返回最终的1x1卷积层,如原始ResNet论文中所述:
但是在Tensorflow示例中,第一个1x1层使用块输出大小,而不是瓶颈大小,因此没有进行有意义的通道缩减。这里的Tensorflow示例真的错了,还是我错过了什么?