Question

除了可以将预训练的嵌入矩阵设置为起点之外，我还希望能够在几个训练时期之后提取该值。这个问题在here中被部分询问，但在对其进行测试后，似乎无法实现。另外，如您所见，在以下实现设置中，权重没有做任何事情，因为嵌入总是使用提供的embedding_initializer

生成的

import numpy as np
import tensorflow as tf
tf.enable_eager_execution()

weights = np.ones((1000,100))*0.5
l = tf.keras.layers.Embedding(1000, 100, weights=[weights])
print(l.weights)
>>> []

# after calling l i can get weights out but they are different from the ones i instanciated 
_ = l(tf.zeros((100,10000)) )
print(l.weights[0].numpy())
>>> array([[ 0.03070325, -0.02488899, -0.04202871, ...,  0.04742295,
     0.02001221, -0.00033263],
   [-0.02426885, -0.02570304, -0.03329657, ...,  0.04899085,
    -0.0036302 ,  0.0438396 ],
   [-0.02653199, -0.04899354,  0.04468955, ...,  0.01906048,
     0.0380123 ,  0.00819932],
   ...,
   [-0.04614455, -0.02689583,  0.03131714, ...,  0.01017661,
     0.01963904,  0.01204284],
   [ 0.03552103, -0.02913042, -0.03126413, ...,  0.02041342,
    -0.04167626,  0.04243967],
   [-0.02114972,  0.0297101 , -0.03390964, ..., -0.01467616,
     0.04121578, -0.02027001]], dtype=float32)

您可以看出这不是np.ones（（1000,100））* 0.5 我想知道我是否正确设置嵌入矩阵...

深入研究TensorFlow.python.keras.layers.embedding.py代码，这是正在发生的事情

@tf_utils.shape_type_conversion
  def build(self, input_shape):
    # Note: most sparse optimizers do not have GPU kernels defined. When
    # building graphs, the placement algorithm is able to place variables on CPU
    # since it knows all kernels using the variable only exist on CPU.
    # When eager execution is enabled, the placement decision has to be made
    # right now. Checking for the presence of GPUs to avoid complicating the
    # TPU codepaths which can handle sparse optimizers.
    if context.executing_eagerly() and context.context().num_gpus():
      with ops.device('cpu:0'):
        self.embeddings = self.add_weight(
            shape=(self.input_dim, self.output_dim),
            initializer=self.embeddings_initializer,
            name='embeddings',
            regularizer=self.embeddings_regularizer,
            constraint=self.embeddings_constraint)
    else:
      self.embeddings = self.add_weight(
          shape=(self.input_dim, self.output_dim),
          initializer=self.embeddings_initializer,
          name='embeddings',
          regularizer=self.embeddings_regularizer,
          constraint=self.embeddings_constraint)
    self.built = True

这意味着无法传递经过训练的嵌入矩阵，因为embedding_initializer是函数名...

如何设置经过训练的嵌入张量并将其从tf.keras.layers.Embedding中取出

0 个答案: