我有一个GloVe的张量流版本:
Xij = tf.placeholder(tf.float32, shape=[None], name="Xij")
wordI = tf.placeholder(tf.int32, shape=[None], name="wordI")
wordIW = tf.Variable(tf.random_uniform([inputSize, embedSize], initRange, -initRange), name="wordIW")
wordIB = tf.Variable(tf.random_uniform([inputSize], initRange, -initRange), name="wordIB")
wi = tf.nn.embedding_lookup([wordIW], wordI, name="wi")
bi = tf.nn.embedding_lookup([wordIB], wordI, name="bi")
wordJ = tf.placeholder(tf.int32, shape=[None], name="wordJ")
wordJW = tf.Variable(tf.random_uniform([outputSize, embedSize], initRange, -initRange), name="wordJW")
wordJB = tf.Variable(tf.random_uniform([outputSize], initRange, -initRange), name="wordJB")
wj = tf.nn.embedding_lookup([wordJW], wordJ, name="wj")
bj = tf.nn.embedding_lookup([wordJB], wordJ, name="bj")
scalingFactor = tf.constant([scalingFactor], name="scalingFactor")
countMax = tf.constant([countMax], name="countMax")
wFactor = tf.minimum(1.0, tf.pow(tf.div(Xij, countMax), scalingFactor))
wiwjProduct = tf.reduce_sum(tf.multiply(wi, wj), 1)
logXij = tf.log(Xij)
dist = tf.square(tf.add_n([wiwjProduct, bi, bj, tf.negative(logXij)]))
loss = tf.reduce_sum(tf.multiply(wFactor, dist), name="loss")
tf.summary.scalar("GloVeLoss", loss)
global_step = tf.Variable(0, trainable=False, name="global_step")
learn_rate = tf.Variable(learn_rate, trainable=False, name="learn_rate")
optimizer = tf.train.GradientDescentOptimizer(learning_rate=learn_rate, name="optimizer").minimize(loss, global_step=global_step)
saver = tf.train.Saver()
我想添加一个新输入(增加输入大小为1并在wordIW
和wordIB
中添加相应的行)。我找不到修改现有图表的正确方法。
ps:我的目标是停止现有变量的渐变,并仅在新输入上优化损失。