caffe,坚持"数据层预取队列空"

时间:2017-04-04 05:57:44

标签: deep-learning caffe

我正在进行caffe的培训阶段。网络是:VGG16 + 8个卷积层+ softmax丢失。由于我的机器,我选择仅在CPU上运行。我的记忆是8G而我的数据集是VOC2007 segmentation images(训练+ val +测试不超过1G内存)。

但是,在输入命令后,程序会在消息" 数据层预取队列为空"很长一段时间(~10小时)。在此期间,系统监视器显示我的CPU使用率为100%,内存为69%。由于我的火车批次只有10,我认为我的机器可以承受的数量,很难理解为什么会发生这种情况?

以下是完成网络初始化和完成Solver脚手架后的输出消息。

I0404 13:08:15.401798 28175 net.cpp:274] Network initialization done.
I0404 13:08:15.401984 28175 solver.cpp:60] Solver scaffolding done.
I0404 13:08:15.402128 28175 caffe.cpp:129] Finetuning from /home/huanghe/caffe-dilation/pretrained/frontend_model.caffemodel
[libprotobuf WARNING google/protobuf/io/coded_stream.cc:537] Reading dangerously large protocol message.  If the message turns out to be larger than 2147483647 bytes, parsing will be halted for security reasons.  To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
[libprotobuf WARNING google/protobuf/io/coded_stream.cc:78] The total number of bytes read was 537388947
I0404 13:08:22.585640 28175 net.cpp:752] Ignoring source layer prob
[libprotobuf WARNING google/protobuf/io/coded_stream.cc:537] Reading dangerously large protocol message.  If the message turns out to be larger than 2147483647 bytes, parsing will be halted for security reasons.  To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
[libprotobuf WARNING google/protobuf/io/coded_stream.cc:78] The total number of bytes read was 537388947
I0404 13:08:23.729429 28175 net.cpp:752] Ignoring source layer prob
I0404 13:08:23.732425 28175 caffe.cpp:219] Starting Optimization
I0404 13:08:23.732533 28175 solver.cpp:279] Solving 
I0404 13:08:23.732580 28175 solver.cpp:280] Learning Rate Policy: step
I0404 13:08:23.833487 28175 solver.cpp:337] Iteration 0, Testing net (#0)
I0404 13:08:23.891206 28175 blocking_queue.cpp:50] Data layer prefetch queue empty

我是新手。在互联网上搜索从来没有帮助我。有谁可以帮我解释一下?非常感谢你!

1 个答案:

答案 0 :(得分:0)

问题已解决。事实证明我走错了路。 caffe只能找不到训练数据并打印数据层预取队列空白"消息。