无法超越第一个时代 - 只是挂起[Keras Transfer Learning Inception]

时间:2017-11-19 23:02:42

标签: python deep-learning keras

我基本上使用了Keras Inception转移学习API教程中的大部分代码,

https://faroit.github.io/keras-docs/2.0.0/applications/#inceptionv3

只需进行一些小改动即可适合我的数据。

我正在使用Tensorflow-gpu 1.4,Windows 7和Keras 2.03(?最新的Keras)。

CODE:

from keras.applications.inception_v3 import InceptionV3
from keras.preprocessing import image
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Model
from keras.layers import Dense, GlobalAveragePooling2D
from keras import backend as K


img_width, img_height = 299, 299
train_data_dir = r'C:\Users\Moondra\Desktop\Keras Applications\data\train'
nb_train_samples = 8
nb_validation_samples = 100 
batch_size = 10
epochs = 5


train_datagen = ImageDataGenerator(
rescale = 1./255,
horizontal_flip = True,
zoom_range = 0.1,
rotation_range=15)



train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size = (img_height, img_width),
batch_size = batch_size, 
class_mode = 'categorical')  #class_mode = 'categorical'


# create the base pre-trained model
base_model = InceptionV3(weights='imagenet', include_top=False)

# add a global spatial average pooling layer
x = base_model.output
x = GlobalAveragePooling2D()(x)
# let's add a fully-connected layer
x = Dense(1024, activation='relu')(x)
# and a logistic layer -- let's say we have 200 classes
predictions = Dense(12, activation='softmax')(x)

# this is the model we will train
model = Model(input=base_model.input, output=predictions)

# first: train only the top layers (which were randomly initialized)
# i.e. freeze all convolutional InceptionV3 layers
for layer in base_model.layers:
    layer.trainable = False

# compile the model (should be done *after* setting layers to non-trainable)
model.compile(optimizer='rmsprop', loss='categorical_crossentropy')

# train the model on the new data for a few epochs
model.fit_generator(
train_generator,
steps_per_epoch = 5,
epochs = epochs)


# at this point, the top layers are well trained and we can start fine-tuning
# convolutional layers from inception V3. We will freeze the bottom N layers
# and train the remaining top layers.

# let's visualize layer names and layer indices to see how many layers
# we should freeze:
for i, layer in enumerate(base_model.layers):
   print(i, layer.name)

# we chose to train the top 2 inception blocks, i.e. we will freeze
# the first 172 layers and unfreeze the rest:
for layer in model.layers[:172]:
   layer.trainable = False
for layer in model.layers[172:]:
   layer.trainable = True

# we need to recompile the model for these modifications to take effect
# we use SGD with a low learning rate
from keras.optimizers import SGD
model.compile(optimizer=SGD(lr=0.0001, momentum=0.9), loss='categorical_crossentropy')

# we train our model again (this time fine-tuning the top 2 inception blocks
# alongside the top Dense layers
model.fit_generator(
train_generator,
steps_per_epoch = 5,
epochs = epochs)

输出(无法通过第一个纪元):

 Epoch 1/5

1/5 [=====>........................] - ETA: 8s - loss: 2.4869
2/5 [===========>..................] - ETA: 3s - loss: 5.5591
3/5 [=================>............] - ETA: 1s - loss: 6.6299

4/5 [=======================>......] - ETA: 0s - loss: 8.4925

它只是挂在这里。

UPDATE:

我使用tensorflow 1.3(降级一个版本)和Keras 2.03(最新的pip版本)创建了一个虚拟环境,但仍然遇到了同样的问题。

更新2

我不认为这是一个记忆问题,好像我改变了纪元内的步骤 - 它会一直运行到最后一步,只是冻结。

一个时代的30个步骤,它将持续到29个。

5个步骤,它将一直运行到第4步然后才挂起。

更新3

还尝试了Keras API中建议的图层249。

5 个答案:

答案 0 :(得分:1)

显然这是一个通过Keras更新修复的错误(但有些人仍然遇到问题)

答案 1 :(得分:1)

似乎大多数冻结问题都是在代码中发生某些错误时发生的。 在my case中,我构建了一个生成器,该生成器在时期结束时引发异常,并且进程停止了。但是没有关于异常的消息,所以我也花一些时间弄清楚发生了什么。

答案 2 :(得分:0)

使用tensorflow.__version__==1.10.1keras.__version__==2.2.2时,我也遇到了同样的问题,对我来说,解决方法是使用keras将其降级为2.2.0 pip3 install -I keras==2.2.0。请注意,这可能会破坏兼容性,并且您可能还需要降级tensorflow

答案 3 :(得分:0)

正如@ thomas-e所提到的,我也对keras / tf兼容性有类似的问题。具体来说,我的配置是:cuda-10.0, cudnn-7, tensorflow_gpu=1.14.0, keras=2.2.5.

通过降级为cuda-9.0, cudnn-7, tensorflow-gpu=1.10.0 and keras=2.2.0

修复了该问题

本文提出了关于不兼容的想法:https://github.com/tensorflow/tensorflow/issues/15604

此外,您可以在以下文章中参考keras和tensorflow兼容性:

  1. https://www.tensorflow.org/install/source#tested_build_configurations
  2. https://docs.floydhub.com/guides/environments/

答案 4 :(得分:0)

保持这些关系解决了问题:

steps_per_epoch = number of train samples/batch_size

validation_steps= number of validation samples/batch_size

更多相同@ https://github.com/keras-team/keras/issues/8595