Question

我正在尝试在tensorflow中将tf.keras与新的AdamW优化器一起使用，并且遇到了问题。该代码的玩具版本如下：

from tensorflow.contrib.opt import AdamWOptimizer
from tensorflow.python.keras.optimizers import TFOptimizer

model = Sequential()
model.add(Dense(2, activation="tanh", input_shape=(3,)))

tfopt = AdamWOptimizer(weight_decay=0.1, learning_rate=.004)
optimizer = TFOptimizer(tfopt)

model.compile(optimizer=optimizer, loss='mean_squared_error')
model.fit(np.random.random((5, 3)),
          np.random.random((5, 2)),
          epochs=5, batch_size=5)

错误如下：

../python3.6/site-packages/tensorflow/python/keras/engine/training.py:1605: in fit
    validation_steps=validation_steps)
../python3.6/site-packages/tensorflow/python/keras/engine/training_arrays.py:153: in fit_loop
    outs = f(ins)
../python3.6/site-packages/tensorflow/python/keras/backend.py:2978: in __call__
    run_metadata=self.run_metadata)
../python3.6/site-packages/tensorflow/python/client/session.py:1399: in __call__
    run_metadata_ptr)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <tensorflow.python.framework.errors_impl.raise_exception_on_not_ok_status object at 0x11ecde550>
type_arg = None, value_arg = None, traceback_arg = None

    def __exit__(self, type_arg, value_arg, traceback_arg):
      try:
        if c_api.TF_GetCode(self.status.status) != 0:
          raise _make_specific_exception(
              None, None,
              compat.as_text(c_api.TF_Message(self.status.status)),
>             c_api.TF_GetCode(self.status.status))
E             tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value training/TFOptimizer/beta2_power
E                [[{{node training/TFOptimizer/beta2_power/read}} = Identity[T=DT_FLOAT, _class=["loc:@training/TFOptimizer/AdamW/Assign"], _device="/job:localhost/replica:0/task:0/device:CPU:0"](training/TFOptimizer/beta2_power)]]

../python3.6/site-packages/tensorflow/python/framework/errors_impl.py:526: FailedPreconditionError

Answer 1

结果证明TFOptimizer在python 3.6中不起作用，但在2.7中起作用。

但是，您实际上不需要使用TFOptimizer。将tensorflow优化器AdamWOptimizer直接插入fit的优化器参数可以很好地运行。

Answer 2

对于我来说，这可以运行，并且对于我的大多数分类目标而言都非常快地收敛到高精度，但是有一个问题：页边距很小，所有分数都接近0.5。另外，一个分类器不收敛。使用亚当，效果会更好。

将tf.Keras与Tensorflow优化器一起使用

2 个答案: