我正在尝试将半精度（bfloat16）与tensor2tensor一起使用
但似乎类型不一致
tensorflow-gpu：1.8.0 张量2张量：1.6.3 的Python：2.7.12 操作系统：Ubuntu 16.04.4 LTS
我运行以下命令来生成数据：
t2t-datagen --t2t_usr_dir ./t2t
--data_dir=./neural_train_data/data
--tmp_dir=./tmp
--problem=translate_mmt
--hparams activation_dtype=bfloat16,weight_dtype=bfloat16,optimizer=Adafactor
问题“ translate_mmt”本质上是“ translate_enfr_wmt32k”
我运行此命令进行训练：
t2t-trainer --t2t_usr_dir ./t2t
--data_dir=./neural_train_data/data
--problem=translate_mmt
--model=transformer
--hparams_set=transformer_base
--output_dir=./neural_model
--local_eval_frequency=500 --train_steps=1000 --worker_gpu=1
--hparams batch_size=3072,activation_dtype=bfloat16,weight_dtype=bfloat16,optimizer=Adafactor
我收到以下错误
INFO:tensorflow:Using variable initializer: uniform_unit_scaling
INFO:tensorflow:Transforming feature 'inputs' with symbol_modality_17717_512.bottom
Traceback (most recent call last):
File "/home/nicola/.local/bin/t2t-trainer", line 32, in <module>
tf.app.run()
File "/home/nicola/.local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 126, in run
_sys.exit(main(argv))
File "/home/nicola/.local/bin/t2t-trainer", line 28, in main
t2t_trainer.main(argv)
File "/home/nicola/.local/lib/python2.7/site-packages/tensor2tensor/bin/t2t_trainer.py", line 353, in main
execute_schedule(exp)
File "/home/nicola/.local/lib/python2.7/site-packages/tensor2tensor/bin/t2t_trainer.py", line 302, in execute_schedule
getattr(exp, FLAGS.schedule)()
File "/home/nicola/.local/lib/python2.7/site-packages/tensorflow/contrib/framework/python/framework/experimental.py", line 64, in new_func
return func(*args, **kwargs)
File "/home/nicola/.local/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/experiment.py", line 773, in continuous_train_and_eval
saving_listeners=self._saving_listeners)
File "/home/nicola/.local/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/experiment.py", line 876, in _call_train
saving_listeners=saving_listeners)
File "/home/nicola/.local/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.py", line 363, in train
loss = self._train_model(input_fn, hooks, saving_listeners)
File "/home/nicola/.local/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.py", line 843, in _train_model
return self._train_model_default(input_fn, hooks, saving_listeners)
File "/home/nicola/.local/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.py", line 856, in _train_model_default
features, labels, model_fn_lib.ModeKeys.TRAIN, self.config)
File "/home/nicola/.local/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.py", line 831, in _call_model_fn
model_fn_results = self._model_fn(features=features, **kwargs)
File "/home/nicola/.local/lib/python2.7/site-packages/tensor2tensor/utils/t2t_model.py", line 922, in wrapping_model_fn
use_tpu=use_tpu)
File "/home/nicola/.local/lib/python2.7/site-packages/tensor2tensor/utils/t2t_model.py", line 972, in estimator_model_fn
logits, losses_dict = model(features) # pylint: disable=not-callable
File "/home/nicola/.local/lib/python2.7/site-packages/tensorflow/python/layers/base.py", line 717, in __call__
outputs = self.call(inputs, *args, **kwargs)
File "/home/nicola/.local/lib/python2.7/site-packages/tensor2tensor/utils/t2t_model.py", line 154, in call
sharded_logits, losses = self.model_fn_sharded(sharded_features)
File "/home/nicola/.local/lib/python2.7/site-packages/tensor2tensor/utils/t2t_model.py", line 209, in model_fn_sharded
sharded_logits, sharded_losses = dp(self.model_fn, datashard_to_features)
File "/home/nicola/.local/lib/python2.7/site-packages/tensor2tensor/utils/expert_utils.py", line 230, in __call__
outputs.append(fns[i](*my_args[i], **my_kwargs[i]))
File "/home/nicola/.local/lib/python2.7/site-packages/tensor2tensor/utils/t2t_model.py", line 233, in model_fn
transformed_features = self.bottom(features)
File "/home/nicola/.local/lib/python2.7/site-packages/tensor2tensor/utils/t2t_model.py", line 275, in bottom
transformed_features[key] = input_modality.bottom(features[key])
File "/home/nicola/.local/lib/python2.7/site-packages/tensor2tensor/layers/modalities.py", line 122, in bottom
return self.bottom_simple(x, "shared", reuse=None)
File "/home/nicola/.local/lib/python2.7/site-packages/tensor2tensor/layers/modalities.py", line 116, in bottom_simple
ret *= tf.expand_dims(tf.to_float(tf.not_equal(x, 0)), -1)
File "/home/nicola/.local/lib/python2.7/site-packages/tensorflow/python/ops/math_ops.py", line 970, in binary_op_wrapper
y = ops.convert_to_tensor(y, dtype=x.dtype.base_dtype, name="y")
File "/home/nicola/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1014, in convert_to_tensor
as_ref=False)
File "/home/nicola/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1104, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/home/nicola/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 947, in _TensorTensorConversionFunction
(dtype.name, t.dtype.name, str(t)))
ValueError: Tensor conversion requested dtype float16 for Tensor with dtype float32: 'Tensor("transformer/parallel_0_5/transformer/symbol_modality_17717_512/shared/ExpandDims:0", shape=(?, ?, 1, 1), dtype=float32, device=/device:GPU:0)'