Question

我构建了一个模型，该模型使用经过预先训练的VGG-16作为基础网络，然后在其上添加几层。

在训练时，我的模型部分由属于预训练的VGG-16的变量组成，即已经初始化的变量（整个模型从SavedModel协议缓冲区加载），部分来自未初始化的未初始化变量我添加的图层。

在开始训练之前，我必须初始化我在预训练的VGG-16之上添加的层的变量。我这样做是通过运行tf.global_variables_initializer()。

以下是问题：

如果tf.global_variables_initializer()初始化所有全局变量，为什么它不会用初始化值覆盖预先训练的权重？

Answer 1

实际上，它确实会覆盖预先训练的权重：

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import backend

session = tf.Session ()
backend.set_session (session)
with session.as_default ():
    net = keras.applications.InceptionV3 (input_shape = (3, 299, 299))
    print ('weights before: ', net.get_layer ('conv2d_1').get_weights ())

    session.run (tf.global_variables_initializer ())

    print ('weights after: ', net.get_layer ('conv2d_1').get_weights ())

输出：

weights before:  [array([[[[ 7.42468759e-02,  7.99097791e-02, -1.20985091e-01, ...,
      -2.42504068e-02, -8.17770883e-03, -7.11583123e-02],
     [ 5.46021610e-02,  1.13289319e-01,  3.99106629e-02, ...,
       2.37093717e-02, -6.01868033e-02, -1.08440826e-02],
     [ 1.00897752e-01,  1.96732566e-01,  8.93706456e-02, ...,
       1.19806841e-01, -9.84039158e-02, -7.20797330e-02],

weights after:  [array([[[[-7.92664364e-02, -5.87778017e-02, -9.76844281e-02, ...,
       9.62431133e-02,  7.16331154e-02, -3.22120935e-02],
     [-6.17715716e-03, -6.95393384e-02,  9.68316942e-02, ...,
       5.01702428e-02,  8.89533758e-02,  9.80506092e-02],
     [-1.89049393e-02, -4.31398787e-02, -9.45695043e-02, ...,
       7.52686858e-02,  7.27956891e-02, -3.26380134e-03],

要解决此问题，请将自定义变量放在单独的集合中：

my_variables_collection = tf.get_collection ('my_variables')

t = tf.Variable (0.0, name='adam_t', trainable=False)
tf.add_to_collection ('my_variables', t)

m = tf.Variable (backend.zeros ((3, 299, 299)), name='adam_m', trainable=False)
tf.add_to_collection ('my_variables', m)

# only initialize variables from my list:
session.run (tf.variables_initializer (my_variables_collection))

Answer 2

在变量初始化程序为我工作之前和之后保存和加载权重

weights=model.get_weights
sess=tf.Session()
K.set_session(sess)
sess.run(tf.global_variables_initializer ())
model.set_weights(weights)
print(np.argmax(sess.run(model(preprocessed_input))))

TensorFlow：在部分加载预训练的权重后使用tf.global_variables_initializer（）

2 个答案: