将tf.estimator.Estimator与save_checkpoint_steps一起使用会导致Tensorboard警告

时间:2018-01-18 08:33:43

标签: python tensorflow tensorboard tensorflow-estimator

使用tf.estimator.train_and_evaluate(...) tf.estimator.Estimator配置tf.contrib.learn.RunConfig(save_checkpoints_steps=10, ...)时,会自动创建CheckpointSaverHook。每次触发时,此CheckpointSaverHook都会将图表和graph_def保存到摘要编写器中(请参阅CheckpointSaverHook.before_run)。

基本代码示例:

estimator = tf.estimator.Estimator(
    model_fn, model_dir, params,
    config=tf.estimator.RunConfig(
        save_checkpoints_steps=100, 
        save_summary_steps=100
    )
)
train_spec = tf.estimator.TrainSpec(train_fn)
eval_spec = tf.estimator.TrainSpec(eval_fn)
tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)

在书面摘要中启动Tensorboard时,它会输出数百个警告,因为摘要中有多个图形定义,我想在启动时会大大减慢它的速度:

W0117 18:47:30.278879 Reloader tf_logging.py:86] Found more than one graph event per run, or there was a metagraph containing a graph_def, as well as one or more graph events.  Overwriting the graph with the newest event.
W0117 18:47:30.279753 Reloader tf_logging.py:86] Found more than one metagraph event per run. Overwriting the metagraph with the newest event.

我发现使用多个图形时可能会出现问题,但对于单个图形,这似乎是不可行的。我们在这里是否正确使用估算器并且这种行为是否有意?我们是否需要编写自定义钩子以防止这种情况发生?

感谢您的建议!

修改:由于评论中的建议,现在出现了问题:https://github.com/tensorflow/tensorflow/issues/17272

0 个答案:

没有答案