Tensorflow Resnet示例:瓶颈大小是否错误?

时间:2018-04-30 11:47:50

标签: python tensorflow deep-learning

以下代码来自https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/learn/resnet.py的Tensorflow Resnet示例:

# Create the bottleneck groups, each of which contains `num_blocks`
# bottleneck groups.
for group_i, group in enumerate(groups):
    for block_i in range(group.num_blocks):
      name = 'group_%d/block_%d' % (group_i, block_i)

      # 1x1 convolution responsible for reducing dimension
      with tf.variable_scope(name + '/conv_in'):
       conv = tf.layers.conv2d(
        net,
        filters=group.num_filters,
        kernel_size=1,
        padding='valid',
        activation=tf.nn.relu)
    conv = tf.layers.batch_normalization(conv, training=training)

  with tf.variable_scope(name + '/conv_bottleneck'):
    conv = tf.layers.conv2d(
        conv,
        filters=group.bottleneck_size,
        kernel_size=3,
        padding='same',
        activation=tf.nn.relu)
    conv = tf.layers.batch_normalization(conv, training=training)

  # 1x1 convolution responsible for restoring dimension
  with tf.variable_scope(name + '/conv_out'):
    input_dim = net.get_shape()[-1].value
    conv = tf.layers.conv2d(
        conv,
        filters=input_dim,
        kernel_size=1,
        padding='valid',
        activation=tf.nn.relu)
    conv = tf.layers.batch_normalization(conv, training=training)

  # shortcut connections that turn the network into its counterpart
  # residual function (identity shortcut)
  net = conv + net

这段代码针对每个块运行,具有给定的输出和瓶颈尺寸。这些块定义为:

  # Configurations for each bottleneck group.
  BottleneckGroup = namedtuple('BottleneckGroup',
                           ['num_blocks', 'num_filters', 'bottleneck_size'])
  groups = [
       BottleneckGroup(3, 128, 32), BottleneckGroup(3, 256, 64),
       BottleneckGroup(3, 512, 128), BottleneckGroup(3, 1024, 256)
  ]

在通常的做法中,据我所知,这些所谓的ResNets瓶颈层首先用1x1内核减少输入通道数,在减少的通道大小中应用更高阶(3x3)的卷积然后恢复到输入通道大小返回最终的1x1卷积层,如原始ResNet论文中所述:

enter image description here

但是在Tensorflow示例中,第一个1x1层使用块输出大小,而不是瓶颈大小,因此没有进行有意义的通道缩减。这里的Tensorflow示例真的错了,还是我错过了什么?

0 个答案:

没有答案