tensorflow TF-slim inceptionv3训练损失曲线很奇怪

Question

从头开始TF-slim inceptionv3列车

我使用slim / train_image_classifier.py在我自己的数据集上训练一个inception_v3模型： python train_image_classifier.py --train_dir = $ {TRAIN_DIR} --dataset_name = mydataset --dataset_split_name = train --dataset_dir = $ {DATASET_DIR} --model_name = inception_v3 --num_clones = 2

损失曲线奇怪，如图所示，它是一条线性减小的直线，有一点凸起部分。

以下是最后的产出，损失每20或30步减少0.0001：

INFO:tensorflow:global step 34590: loss = 0.5359 (1.17 sec/step)
INFO:tensorflow:global step 34600: loss = 0.5358 (1.15 sec/step)
INFO:tensorflow:global step 34590: loss = 0.5359 (1.17 sec/step)
INFO:tensorflow:global step 34600: loss = 0.5358 (1.15 sec/step)
INFO:tensorflow:global step 34610: loss = 0.5358 (1.17 sec/step)
INFO:tensorflow:global step 34620: loss = 0.5357 (1.12 sec/step)
INFO:tensorflow:global step 34630: loss = 0.5357 (1.16 sec/step)
INFO:tensorflow:global step 34640: loss = 0.5356 (1.16 sec/step)
INFO:tensorflow:global step 34650: loss = 0.5356 (1.16 sec/step)
INFO:tensorflow:global step 34660: loss = 0.5355 (1.15 sec/step)
INFO:tensorflow:global step 34670: loss = 0.5355 (1.15 sec/step)
INFO:tensorflow:global step 34680: loss = 0.5355 (1.18 sec/step)
INFO:tensorflow:global step 34690: loss = 0.5354 (1.17 sec/step)
INFO:tensorflow:global step 34700: loss = 0.5354 (1.15 sec/step)
INFO:tensorflow:global step 34710: loss = 0.5353 (1.15 sec/step)
INFO:tensorflow:global step 34720: loss = 0.5353 (2.25 sec/step)
INFO:tensorflow:global step 34730: loss = 0.5353 (2.22 sec/step)
INFO:tensorflow:global step 34740: loss = 0.5352 (1.16 sec/step)
INFO:tensorflow:global step 34750: loss = 0.5352 (1.16 sec/step)
INFO:tensorflow:global step 34760: loss = 0.5351 (1.18 sec/step)
INFO:tensorflow:global step 34770: loss = 0.5351 (1.15 sec/step)
INFO:tensorflow:global step 34780: loss = 0.5350 (1.17 sec/step)
INFO:tensorflow:global step 34790: loss = 0.5350 (1.15 sec/step)
INFO:tensorflow:global step 34800: loss = 0.5349 (1.12 sec/step)
INFO:tensorflow:global step 34810: loss = 0.5349 (1.12 sec/step)
INFO:tensorflow:global step 34820: loss = 0.5349 (1.16 sec/step)
INFO:tensorflow:global step 34830: loss = 0.5348 (1.16 sec/step)
INFO:tensorflow:global step 34840: loss = 0.5348 (1.18 sec/step)
INFO:tensorflow:global step 34850: loss = 0.5347 (1.12 sec/step)
INFO:tensorflow:global step 34860: loss = 0.5347 (1.12 sec/step)
INFO:tensorflow:global step 34870: loss = 0.5347 (1.18 sec/step)
INFO:tensorflow:global step 34880: loss = 0.5346 (1.13 sec/step)
INFO:tensorflow:global step 34890: loss = 0.5346 (1.18 sec/step)
INFO:tensorflow:global step 34900: loss = 0.5345 (1.16 sec/step)
INFO:tensorflow:global step 34910: loss = 0.5345 (1.15 sec/step)
INFO:tensorflow:global step 34920: loss = 0.5344 (1.17 sec/step)
INFO:tensorflow:global step 34930: loss = 0.5344 (1.14 sec/step)
INFO:tensorflow:global step 34940: loss = 0.5344 (1.15 sec/step)
INFO:tensorflow:global step 34950: loss = 0.5343 (1.14 sec/step)
INFO:tensorflow:global step 34960: loss = 0.5343 (1.17 sec/step)  

mydataset.py与flowers.py相同，但是：

SPLITS_TO_SIZES = {'train': 18000000, 'validation': 400000}
 _NUM_CLASSES = 4

这是正常的吗？谢谢你的帮助。

Answer 1

您正在绘制n个步骤后的总损失图（在您的训练中可能是number_of_steps，如果您使用tf.contrib.slim训练方法），而记录的丢失是每10个步骤。 希望这可以帮助！