Question

我正在尝试重新创建模型并重新生成其发布的结果。我使用TF 2.0，并且我认为该模型是使用Theano后端编码的。该模型来自github中的repo，用于上下文。

我也不使用tensorflow-gpu，因为它与我的硬件设置不兼容。

无论如何，起初我在尝试加载其权重甚至模型时遇到了很多错误。当我意识到保存/加载功能刚刚搞砸时，我继续尝试加载和训练模型。由于我使用Tensorflow，因此我修改了代码以使用'channels_last'格式或NHWC，正如错误所言。

这是我的导入列表：

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals

from tensorflow import keras

import cv2
import os
import pathlib
import shutil

import matplotlib.pyplot as plt
import numpy as np
import tensorflow as tf

修改后的模型：

def createModel():
    model = keras.models.Sequential()
    model.add(keras.layers.Lambda(norm_input, input_shape=(28,28,1), output_shape=(28,28,1)))
    model.add(keras.layers.Conv2D(32, (3,3)))
    model.add(keras.layers.LeakyReLU())
    model.add(keras.layers.BatchNormalization(axis=1))
    model.add(keras.layers.Conv2D(32, (3,3)))
    model.add(keras.layers.LeakyReLU())
    model.add(keras.layers.MaxPooling2D())
    model.add(keras.layers.BatchNormalization(axis=1))
    model.add(keras.layers.Conv2D(64, (3,3)))
    model.add(keras.layers.LeakyReLU())
    model.add(keras.layers.BatchNormalization(axis=1))
    model.add(keras.layers.Conv2D(64, (3,3)))
    model.add(keras.layers.LeakyReLU())
    model.add(keras.layers.MaxPooling2D())
    model.add(keras.layers.Flatten())
    model.add(keras.layers.BatchNormalization())
    model.add(keras.layers.Dense(512))
    model.add(keras.layers.LeakyReLU())
    model.add(keras.layers.BatchNormalization())
    model.add(keras.layers.Dropout(0.3))
    model.add(keras.layers.Dense(10, activation='softmax'))

    model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

    return model

我如何加载和预处理MNIST数据集：

(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()

test_labels = y_test

x_train = x_train.reshape(x_train.shape[0], 28, 28, 1)
x_test = x_test.reshape(x_test.shape[0], 28, 28, 1)

x_train = x_train.astype(np.float32)
x_test = x_test.astype(np.float32)

x_train /= 255
x_test /= 255

y_train = keras.utils.to_categorical(y_train, 10)
y_test = keras.utils.to_categorical(y_test, 10)

ImageDataGenerator：

    gen = keras.preprocessing.image.ImageDataGenerator(
            rotation_range=12, 
            width_shift_range=0.1, 
            shear_range=0.3,
            height_shift_range=0.1, 
            zoom_range=0.1, 
            data_format='channels_last')

最后是训练模型的功能：

def fit_model(m):
    m.fit_generator(batches, steps_per_epoch=steps_per_epoch, epochs=1, verbose=0,
                   validation_data=test_batches, validation_steps=validation_steps)
    m.optimizer.lr = 0.1
    m.fit_generator(batches, steps_per_epoch=steps_per_epoch, epochs=4, verbose=0,
                   validation_data=test_batches, validation_steps=validation_steps)
    m.optimizer.lr = 0.01
    m.fit_generator(batches, steps_per_epoch=steps_per_epoch, epochs=12, verbose=0,
                   validation_data=test_batches, validation_steps=validation_steps)
    m.optimizer.lr = 0.001
    m.fit_generator(batches, steps_per_epoch=steps_per_epoch, epochs=18, verbose=0,
                   validation_data=test_batches, validation_steps=validation_steps)
    return m

最后一个代码段是错误指向的位置，但是我不知道到底是什么与图像格式有关。具体来说，它指向第三行或以validation_data=...开头的行。

完整错误是：

Component function execution failed: Internal: The CPU implementation of FusedBatchNorm only supports NHWC tensor format for now.
     [[{{node batch_normalization_v2/cond/then/_0/FusedBatchNorm}}]]

和追溯：

Traceback (most recent call last):
  File "model3.py", line 113, in <module>
    m = fit_model(createModel())
  File "model3.py", line 52, in fit_model
    validation_data=test_batches, validation_steps=validation_steps)
  File "/home/ren/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 1515, in fit_generator
    steps_name='steps_per_epoch')
  File "/home/ren/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_generator.py", line 257, in model_iteration
    batch_outs = batch_function(*batch_data)
  File "/home/ren/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 1259, in train_on_batch
    outputs = self._fit_function(ins)  # pylint: disable=not-callable
  File "/home/ren/.local/lib/python3.6/site-packages/tensorflow/python/keras/backend.py", line 3217, in __call__
    outputs = self._graph_fn(*converted_inputs)
  File "/home/ren/.local/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 558, in __call__
    return self._call_flat(args)
  File "/home/ren/.local/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 627, in _call_flat
    outputs = self._inference_function.call(ctx, args)
  File "/home/ren/.local/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 415, in call
    ctx=ctx)
  File "/home/ren/.local/lib/python3.6/site-packages/tensorflow/python/eager/execute.py", line 66, in quick_execute
    six.raise_from(core._status_to_exception(e.code, message), None)
  File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.InternalError: The CPU implementation of FusedBatchNorm only supports NHWC tensor format for now.
     [[{{node batch_normalization_v2/cond/then/_0/FusedBatchNorm}}]] [Op:__inference_keras_scratch_graph_3602]

我希望当我在代码顶部添加行tf.keras.backend.set_image_data_format('channels_last')时，它会得到修复。为了更好的衡量，我什至在前面提到的ImageDataGenerator中也加入了同样的论点。因此，老实说，我不知道我错过了什么或哪里出错了。

Answer 1

免责声明

我只是该领域的新手，并且此答案仅基于我在文档中阅读的内容。我对可能的影响及其对模型的影响程度不完全理解；最糟糕的是，它可能会影响其性能和准确性。最好的情况下，此解决方案只能视为临时补丁以抑制错误，至少要等到其他人确认为止。始终欢迎任何确认，澄清，建议和替代解决方案。

从BatchNormalization上的Keras documentation：

轴：整数，应归一化的轴（通常是要素轴）。例如，在包含Conv2D的{{1}}图层之后，在data_format="channels_first"中设置axis=1。

由于我使用BatchNormalization或channels_last作为图像格式，因此只需在固定它的参数中将NHWC等同于axis。要清楚：

-1

解决了我的问题。

“ FusedBatchNorm的CPU实现目前仅支持NHWC张量格式。”

1 个答案: