如何解决在目标检测中延迟训练的问题

时间:2019-06-14 09:10:21

标签: object detection

我正在google合作实验室研究对象检测。这样我就可以从自己的图像类成功获得结果。
但是培训存在一些问题。我只是使用相同的代码训练了另一个图像集。但是火车经过了很长时间的加工。我认为某些过程的步骤中存在问题。

你能帮我吗?

所有先前的过程都可以成功完成。我还使用其他图像集验证了代码。

%cd ~/datalab

!python ~/models/research/object_detection/model_main.py \
    --pipeline_config_path=/root/models/research/object_detection/samples/configs/faster_rcnn_inception_v2_pets.config \
    --model_dir=/root/datalab/trained \
    --alsologtostderr \
    --num_train_steps=3000 \
    --num_eval_steps=500

I just do this process. It is basic tutorial code for the object detection.

2019-06-14 08:46:56.402112: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2019-06-14 08:46:56.588324: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0
2019-06-14 08:46:56.666950: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0
2019-06-14 08:46:56.688098: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0
2019-06-14 08:46:56.882239: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0
2019-06-14 08:46:56.984322: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0
2019-06-14 08:46:57.329048: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2019-06-14 08:46:57.329351: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-06-14 08:46:57.329942: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-06-14 08:46:57.330337: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0
2019-06-14 08:46:57.332937: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2019-06-14 08:46:57.335227: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-06-14 08:46:57.335267: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187]      0 
2019-06-14 08:46:57.335282: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0:   N 
2019-06-14 08:46:57.337783: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-06-14 08:46:57.338275: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-06-14 08:46:57.338663: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:40] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.
2019-06-14 08:46:57.338709: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14202 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5)
2019-06-14 08:47:01.278323: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set.  If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU.  To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
I0614 08:47:01.356435 140064085112704 session_manager.py:500] Running local_init_op.
I0614 08:47:01.522893 140064085112704 session_manager.py:502] Done running local_init_op.
I0614 08:47:08.389612 140064085112704 basic_session_run_hooks.py:606] Saving checkpoints for 0 into /root/datalab/trained/model.ckpt.
2019-06-14 08:47:13.301305: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0

输出结尾是矩形框,表示“处理中”

我认为这不是错误。但是我没有结果。

有人遇到过这些问题吗?

0 个答案:

没有答案