Question

我正在使用TPU上的TensorFlow和Keras，我需要加载预训练的BERT模型并将其转换为TPU模型。我首先加载模型，它运行良好。

def _load_bert(self):

    logging.info('Loading BERT from %s', self.bert_path)
    config_path = os.path.join(self.bert_path, 'bert_config.json')
    checkpoint_path = os.path.join(self.bert_path, 'bert_model.ckpt')

    bert = keras_bert.load_trained_model_from_checkpoint(
        config_path,
        checkpoint_path
    )

    self._bert = bert
    self._seq_length = int(self._bert.input[0].shape[1])

然后我尝试执行以下操作将其转换为TPU模型：

self._bert = tensorflow.contrib.tpu.keras_to_tpu_model(
    self._bert,
    strategy=tensorflow.contrib.tpu.TPUDistributionStrategy(
        tensorflow.contrib.cluster_resolver.TPUClusterResolver(
            self.tpu, zone=self.tpu_zone, project=self.gcp_project
        )
    )
)

然后由于某种原因，出现以下错误：

ValueError: ('Expected `model` argument to be a `Model` instance, got ', <keras.engine.training.Model object at xxxxxxxxxx >)

当我不将其转换为TPU模型时，它可以工作。有人有线索吗？

Answer 1

尝试在运行或训练它之前使用策略编译keras.engine.training.Model对象。

resolver = tf.contrib.cluster_resolver.TPUClusterResolver(self.tpu, 
                                                           zone=self.tpu_zone, 
                                                           project=self.gcp_project)
tf.contrib.distribute.initialize_tpu_system(resolver)
strategy = tf.contrib.distribute.TPUStrategy(resolver)
with strategy.scope():
  model = self._bert
  model.compile(optimizer=tf.keras.optimizers.Adagrad(learning_rate=0.1), 
                 loss='mean_squared_error', metrics=['acc','mse'])

使用Keras在TPU上加载预训练的BERT时出错

1 个答案: