训练数据在model.fit_generator()中面临错误

时间:2020-09-06 17:58:42

标签: tensorflow deep-learning computer-vision conv-neural-network cnn

我正在使用Tensorflow训练10类不同图像的多类CNN模型。总训练图像为6000,测试图像为1600。当我尝试训练模型时,我正面临错误。以下是我的代码:

import os
import random
import tensorflow as tf
from tensorflow.keras.optimizers import RMSprop
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from shutil import copyfile

print(len(os.listdir('C:/Users/shweta/Desktop/characters/test'))) #test set
print(len(os.listdir('C:/Users/shweta/Desktop/characters/train'))) #train set

TRAINING_DIR = "C:/Users/shweta/Desktop/characters/train/"

train_datagen = ImageDataGenerator(rescale=1./255,
                                   shear_range=0.2,
                                   zoom_range=0.2,
                                   horizontal_flip=True)

train_generator = train_datagen.flow_from_directory(TRAINING_DIR,
                                                    batch_size=60,
                                                    class_mode='categorical',
                                                    target_size=(64,64))

VALIDATION_DIR = "C:/Users/shweta/Desktop/characters/test/"

validation_datagen = ImageDataGenerator(rescale=1./255)
validation_generator = validation_datagen.flow_from_directory(VALIDATION_DIR,
                                                              batch_size=64,
                                                              class_mode='categorical',
                                                              target_size=(64,64))



model = tf.keras.models.Sequential([
    tf.keras.layers.Conv2D(32, (3, 3), activation='relu', input_shape=(64, 64, 3)),
    tf.keras.layers.MaxPooling2D(2, 2),
    tf.keras.layers.Conv2D(32, (3, 3), activation='relu'),
    tf.keras.layers.MaxPooling2D(2, 2),
    tf.keras.layers.Conv2D(32, (3, 3), activation='relu'),
    tf.keras.layers.MaxPooling2D(2, 2),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(128, activation='relu'),
    
    tf.keras.layers.Dense(10, activation='softmax')
])

model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['acc'])

model.summary()


history = model.fit_generator(train_generator,
                              validation_data=validation_generator,
                              steps_per_epoch=100,
                              epochs=25,
                              validation_steps=25)
                          

我在model.fit_generator中遇到以下错误:

        WARNING:tensorflow:From C:\Users\shweta\.spyder-py3\temp.py:55: Model.fit_generator (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.
Instructions for updating:
Please use Model.fit, which supports generators.
Epoch 1/25
  4/100 [>.............................] - ETA: 2:03 - loss: 2.3212 - acc: 0.1208Traceback (most recent call last):

  File "C:\Users\shweta\.spyder-py3\temp.py", line 55, in <module>
    validation_steps=25)

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\util\deprecation.py", line 324, in new_func
    return func(*args, **kwargs)

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\keras\engine\training.py", line 1479, in fit_generator
    initial_epoch=initial_epoch)

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\keras\engine\training.py", line 66, in _method_wrapper
    return method(self, *args, **kwargs)

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\keras\engine\training.py", line 848, in fit
    tmp_logs = train_function(iterator)

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\eager\def_function.py", line 580, in __call__
    result = self._call(*args, **kwds)

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\eager\def_function.py", line 611, in _call
    return self._stateless_fn(*args, **kwds)  # pylint: disable=not-callable

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\eager\function.py", line 2420, in __call__
    return graph_function._filtered_call(args, kwargs)  # pylint: disable=protected-access

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\eager\function.py", line 1665, in _filtered_call
    self.captured_inputs)

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\eager\function.py", line 1746, in _call_flat
    ctx, args, cancellation_manager=cancellation_manager))

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\eager\function.py", line 598, in call
    ctx=ctx)

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\eager\execute.py", line 60, in quick_execute
    inputs, attrs, num_outputs)

UnknownError:  UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x00000180610E2E28>
Traceback (most recent call last):

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\ops\script_ops.py", line 243, in __call__
    ret = func(*args)

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\autograph\impl\api.py", line 309, in wrapper
    return func(*args, **kwargs)

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py", line 785, in generator_py_func
    values = next(generator_state.get_iterator(iterator_id))

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\keras\engine\data_adapter.py", line 801, in wrapped_generator
    for data in generator_fn():

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\keras\engine\data_adapter.py", line 932, in generator_fn
    yield x[i]

  File "C:\Users\shweta\anaconda3\lib\site-packages\keras_preprocessing\image\iterator.py", line 65, in __getitem__
    return self._get_batches_of_transformed_samples(index_array)

  File "C:\Users\shweta\anaconda3\lib\site-packages\keras_preprocessing\image\iterator.py", line 230, in _get_batches_of_transformed_samples
    interpolation=self.interpolation)

  File "C:\Users\shweta\anaconda3\lib\site-packages\keras_preprocessing\image\utils.py", line 114, in load_img
    img = pil_image.open(io.BytesIO(f.read()))

  File "C:\Users\shweta\anaconda3\lib\site-packages\PIL\Image.py", line 2862, in open
    "cannot identify image file %r" % (filename if filename else fp)

PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x00000180610E2E28>


     [[{{node PyFunc}}]]
     [[IteratorGetNext]] [Op:__inference_train_function_877]

Function call stack:
train_function

请帮助我解决此问题。预先感谢。

2 个答案:

答案 0 :(得分:0)

请将您的softmax输出更改为10,而不是将11作为类别数。 如果您希望添加否定类,然后在训练和测试数据集中再添加一个文件夹。

tf.keras.layers.Dense(10, activation='softmax')

答案 1 :(得分:0)

一些观察。您有6000张训练图像,并且将批次大小指定为64,每个时期的步数=200。200 X 64 = 12,800,因此您将经历每个时期两次的训练集。将批次大小设置为60,将每个时期的步数设置为100,您将每个时期进行一次培训。对于验证数据,您有类似的问题。您只希望每个时期通过一次验证集。如果有1600张验证图像且批处理大小= 64,则1600/64 = 25,因此设置validation_steps = 25。现在,我不确定这是否可以解决您的问题。试试看,看看是否可以解决。如果不是,我怀疑您的输入数据集中可能存在无效内容。我开发了一个脚本来检查输入目录,以确保它们具有允许的扩展名,并且实际上是好的图像文件。脚本位于here.

中的答案中