我正在使用this回购，以使用预训练的BERT模型进行多标签分类。我有一个预训练的BERT模型，我想对其分类问题进行微调。运行提供的多标签分类脚本（run_bluebert_multi_labels.py）时，出现以下错误。
ValueError: Shape of variable bert/embeddings/word_embeddings:0 ((28996, 768)) doesn't match with shape of tensor bert/embeddings/word_embeddings ([30522, 768]) from checkpoint reader.
这是回溯：
WARNING:tensorflow:From bluebert/run_bluebert_multi_labels.py:163: The name tf.gfile.Open is deprecated. Please use tf.io.gfile.GFile instead.
WARNING:tensorflow:Estimator's model_fn (<function model_fn_builder.<locals>.model_fn at 0x7f9756ef84d0>) includes params argument, but params are not passed to Estimator.
INFO:tensorflow:Using config: {'_model_dir': './output', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': 1000, '_save_checkpoints_secs': None, '_session_config': allow_soft_placement: true
graph_options {
rewrite_options {
meta_optimizer_iterations: ONE
}
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': None, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f9756796310>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1, '_tpu_config': TPUConfig(iterations_per_loop=1000, num_shards=8, num_cores_per_replica=None, per_host_input_for_training=3, tpu_job_name=None, initial_infeed_sleep_secs=None, input_partition_dims=None, eval_training_input_configuration=2), '_cluster': None}
INFO:tensorflow:_TPUContext: eval_on_tpu True
WARNING:tensorflow:eval_on_tpu ignored because use_tpu is False.
inside file based:
WARNING:tensorflow:From bluebert/run_bluebert_multi_labels.py:351: The name tf.python_io.TFRecordWriter is deprecated. Please use tf.io.TFRecordWriter instead.
WARNING:tensorflow:From bluebert/run_bluebert_multi_labels.py:356: The name tf.logging.info is deprecated. Please use tf.compat.v1.logging.info instead.
INFO:tensorflow:Writing example 0 of 20137
INFO:tensorflow:Writing example 10000 of 20137
INFO:tensorflow:Writing example 20000 of 20137
INFO:tensorflow:***** Running training *****
INFO:tensorflow: Num examples = 20137
INFO:tensorflow: Batch size = 32
INFO:tensorflow: Num steps = 1887
WARNING:tensorflow:From bluebert/run_bluebert_multi_labels.py:389: The name tf.FixedLenFeature is deprecated. Please use tf.io.FixedLenFeature instead.
WARNING:tensorflow:From /home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
WARNING:tensorflow:From bluebert/run_bluebert_multi_labels.py:425: map_and_batch (from tensorflow.contrib.data.python.ops.batching) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.experimental.map_and_batch(...)`.
WARNING:tensorflow:From /home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow/contrib/data/python/ops/batching.py:273: map_and_batch (from tensorflow.python.data.experimental.ops.batching) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.map(map_func, num_parallel_calls)` followed by `tf.data.Dataset.batch(batch_size, drop_remainder)`. Static tf.data optimizations will take care of using the fused implementation.
WARNING:tensorflow:From bluebert/run_bluebert_multi_labels.py:398: The name tf.parse_single_example is deprecated. Please use tf.io.parse_single_example instead.
WARNING:tensorflow:From bluebert/run_bluebert_multi_labels.py:405: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Running train on CPU
INFO:tensorflow:*** Features ***
INFO:tensorflow: name = input_ids, shape = (32, 128)
INFO:tensorflow: name = input_mask, shape = (32, 128)
INFO:tensorflow: name = is_real_example, shape = (32,)
INFO:tensorflow: name = label_ids, shape = (32, 19)
INFO:tensorflow: name = segment_ids, shape = (32, 128)
WARNING:tensorflow:From /home/aditya/Projects/RD/ncbi_bluebert/bert/modeling.py:172: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.
WARNING:tensorflow:From /home/aditya/Projects/RD/ncbi_bluebert/bert/modeling.py:411: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.
WARNING:tensorflow:From /home/aditya/Projects/RD/ncbi_bluebert/bert/modeling.py:492: The name tf.assert_less_equal is deprecated. Please use tf.compat.v1.assert_less_equal instead.
WARNING:tensorflow:From /home/aditya/Projects/RD/ncbi_bluebert/bert/modeling.py:359: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
WARNING:tensorflow:From /home/aditya/Projects/RD/ncbi_bluebert/bert/modeling.py:673: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.dense instead.
WARNING:tensorflow:From /home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow/python/ops/nn_impl.py:180: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From bluebert/run_bluebert_multi_labels.py:546: The name tf.train.init_from_checkpoint is deprecated. Please use tf.compat.v1.train.init_from_checkpoint instead.
ERROR:tensorflow:Error recorded from training_loop: Shape of variable bert/embeddings/word_embeddings:0 ((28996, 768)) doesn't match with shape of tensor bert/embeddings/word_embeddings ([30522, 768]) from checkpoint reader.
INFO:tensorflow:training_loop marked as finished
WARNING:tensorflow:Reraising captured error
Traceback (most recent call last):
File "bluebert/run_bluebert_multi_labels.py", line 876, in <module>
main(args)
File "bluebert/run_bluebert_multi_labels.py", line 781, in main
estimator.train(input_fn=train_input_fn, max_steps=num_train_steps)
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 2876, in train
rendezvous.raise_errors()
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/tpu/error_handling.py", line 131, in raise_errors
six.reraise(typ, value, traceback)
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/six.py", line 696, in reraise
raise value
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 2871, in train
saving_listeners=saving_listeners)
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 367, in train
loss = self._train_model(input_fn, hooks, saving_listeners)
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1158, in _train_model
return self._train_model_default(input_fn, hooks, saving_listeners)
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1188, in _train_model_default
features, labels, ModeKeys.TRAIN, self.config)
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 2709, in _call_model_fn
config)
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1146, in _call_model_fn
model_fn_results = self._model_fn(features=features, **kwargs)
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 2967, in _model_fn
features, labels, is_export_mode=is_export_mode)
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 1549, in call_without_tpu
return self._call_model_fn(features, labels, is_export_mode=is_export_mode)
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 1867, in _call_model_fn
estimator_spec = self._model_fn(features=features, **kwargs)
File "bluebert/run_bluebert_multi_labels.py", line 546, in model_fn
tf.train.init_from_checkpoint(init_checkpoint, assignment_map)
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow/python/training/checkpoint_utils.py", line 291, in init_from_checkpoint
init_from_checkpoint_fn)
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow/python/distribute/distribute_lib.py", line 1684, in merge_call
return self._merge_call(merge_fn, args, kwargs)
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow/python/distribute/distribute_lib.py", line 1691, in _merge_call
return merge_fn(self._strategy, *args, **kwargs)
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow/python/training/checkpoint_utils.py", line 286, in <lambda>
ckpt_dir_or_file, assignment_map)
File "/home/aditya/anaconda3/envs/RD/lib/python3.7/site-packages/tensorflow/python/training/checkpoint_utils.py", line 329, in _init_from_checkpoint
tensor_name_in_ckpt, str(variable_map[tensor_name_in_ckpt])
ValueError: Shape of variable bert/embeddings/word_embeddings:0 ((28996, 768)) doesn't match with shape of tensor bert/embeddings/word_embeddings ([30522, 768]) from checkpoint reader.
我认为错误是因为我的预训练模型的唱头尺寸为28996，而BERT Base的唱头尺寸为30522。但是为什么会这样？