用于自定义数据集的Tensorflow对象检测API - 在培训期间被杀死

时间：2017-09-28 19:42:13

标签： python linux tensorflow object-detection-api

对自定义数据集（店铺前端图像）进行对象检测的训练，对于单个类（总共285个图像），在CPU上本地运行，8GB RAM在几步后获得 kill 。

我跟随此blog作为参考。

这是控制台日志

(tensorflow) rajaram@rajaram-Lenovo-ideapad-110-15ISK:~/tensorflow/models$ python object_detection/train.py \
>     --logtostderr \
>     --pipeline_config_path=/home/rajaram/tensorflow/models/object_detection/models/sf_od_model/ssd_mobilenet_v1_sf_train.config \
>     --train_dir=/home/rajaram/tensorflow/models/object_detection/models/sf_od_model/train
INFO:tensorflow:Summary name Learning Rate is illegal; using Learning_Rate instead.
WARNING:tensorflow:From /home/rajaram/tensorflow/models/object_detection/meta_architectures/ssd_meta_arch.py:607: all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02.
Instructions for updating:
Please use tf.global_variables instead.
INFO:tensorflow:Summary name /clone_loss is illegal; using clone_loss instead.
2017-09-26 22:15:08.121785: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-26 22:15:08.122313: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-26 22:15:08.123308: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-09-26 22:15:08.124144: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-26 22:15:08.124658: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2017-09-26 22:15:08.953929: I tensorflow/core/common_runtime/simple_placer.cc:697] Ignoring device specification /device:GPU:0 for node 'prefetch_queue_Dequeue' because the input edge from 'prefetch_queue' is a reference connection and already has a device field set to /device:CPU:0
INFO:tensorflow:Restoring parameters from /home/rajaram/tensorflow/models/object_detection/models/sf_od_model/ssd_mobilenet_v1_coco_11_06_2017/model.ckpt
INFO:tensorflow:Starting Session.
INFO:tensorflow:Saving checkpoint to path /home/rajaram/tensorflow/models/object_detection/models/sf_od_model/train/model.ckpt
INFO:tensorflow:Starting Queues.
INFO:tensorflow:global_step/sec: 0
INFO:tensorflow:Recording summary at step 0.
INFO:tensorflow:global_step/sec: 0
INFO:tensorflow:global_step/sec: 0
INFO:tensorflow:global_step/sec: 0
INFO:tensorflow:Recording summary at step 0.
INFO:tensorflow:Recording summary at step 0.
INFO:tensorflow:Recording summary at step 0.
INFO:tensorflow:global_step/sec: 0
INFO:tensorflow:Saving checkpoint to path /home/rajaram/tensorflow/models/object_detection/models/sf_od_model/train/model.ckpt
INFO:tensorflow:global_step/sec: 0
INFO:tensorflow:Recording summary at step 1.
INFO:tensorflow:Recording summary at step 1.
INFO:tensorflow:global_step/sec: 0.00238991
INFO:tensorflow:global_step/sec: 0
INFO:tensorflow:Recording summary at step 1.
INFO:tensorflow:global_step/sec: 0
INFO:tensorflow:global step 1: loss = 14.4365 (801.196 sec/step)
INFO:tensorflow:Recording summary at step 1.
INFO:tensorflow:Recording summary at step 1.
INFO:tensorflow:global_step/sec: 0
INFO:tensorflow:Recording summary at step 1.
INFO:tensorflow:global step 2: loss = 12.9940 (173.981 sec/step)
INFO:tensorflow:Recording summary at step 2.
INFO:tensorflow:Recording summary at step 3.
INFO:tensorflow:global step 3: loss = 12.4866 (166.656 sec/step)
INFO:tensorflow:Saving checkpoint to path /home/rajaram/tensorflow/models/object_detection/models/sf_od_model/train/model.ckpt
INFO:tensorflow:Recording summary at step 3.
INFO:tensorflow:Saving checkpoint to path /home/rajaram/tensorflow/models/object_detection/models/sf_od_model/train/model.ckpt
INFO:tensorflow:global step 4: loss = 11.2386 (162.260 sec/step)
INFO:tensorflow:Recording summary at step 4.
INFO:tensorflow:Recording summary at step 4.
INFO:tensorflow:Recording summary at step 5.
INFO:tensorflow:global step 5: loss = 10.8210 (416.903 sec/step)
INFO:tensorflow:Recording summary at step 5.
Killed
(tensorflow) rajaram@rajaram-Lenovo-ideapad-110-15ISK:~/tensorflow/models$

我的想法和问题

1）图片尺寸是个问题？ - 我的图像分布如下： ＆lt; = 400x300（5％），400x300＆amp; 640x480（22％），640x480和640x480之间800x600（63％）和＆gt; 800x600（22％）。虽然尺寸大约为400x300的图像足以识别商店，但我的数据集中存在较大分辨率的偏差，因为下一步是在这些板上进行文本识别。

  • 这是正确的吗？
  • 我应该将图像调整到较小的尺寸（如果是 - 尺寸是多好）并在重新开始整个过程​​之前重新进行注释？

我可以训练牛津-IIIT宠物数据（大约7.9k图像 - 花了大约13个小时）进行2000步（在配置文件的train_config部分中num_steps = 2000），没有崩溃或被杀死。所以，我认为只有285个图像应该可以在CPU本身上运行。

2）交换内存是一个问题吗？ - 我还检查了其他帖子（increase swap space suggestion，no follow-up＆amp; another increase swap memory suggestion）虽然我可以在我目前的系统设置上训练牛津IIIT宠物数据集，但只训练285张图像不应该杀死这个过程。

  • 我的想法是否正确？
  • 如果没有，这确实是一个解决方案，那么我需要指针和明确的步骤来做到这一点。

我想知道出了什么问题并让它在本地运行。我希望我已经提供了足够的信息来获得帮助。如果没有，请告诉我需要什么。

                         ---------------------------

系统信息

  1. 您正在使用的模型的顶级目录是什么： tensorflow / models（尚未更新到新的文件夹结构）
  2. 我是否编写了自定义代码（与使用TensorFlow中提供的库存示例脚本相反）：是 - 最小的更改（Dat Trans's template for my own data set - Github之后）
  3. OS平台和发行版（例如，Linux Ubuntu 16.04）： Ubuntu 16.04.3 LTS
  4. 从（源代码或二进制代码）安装的TensorFlow：二进制文件（进入虚拟环境）
  5. TensorFlow版本（使用下面的命令）： 1.3.0
  6. Bazel版本（如果从源代码编译）： NA
  7. CUDA / cuDNN版本： NA
  8. GPU型号和内存： NA
  9. 要重现的确切命令： python object_detection / train.py --logtostderr --pipeline_config_path = / home / rajaram / tensorflow / models / object_detection / models / sf_od_model / ssd_mobilenet_v1_sf_train.config --train_dir = /家庭/拉贾拉姆/ tensorflow /模型/ object_detection /模型/ sf_od_model /列车

0 个答案:

没有答案
相关问题
最新问题