我构建了一个caffe网络+求解器(用于二进制分类),当我运行代码(并尝试训练网络)时,我看到了这个错误:
I0914 20:03:01.362612 4024 solver.cpp:280] Learning Rate Policy: step
I0914 20:03:01.367985 4024 solver.cpp:337] Iteration 0, Testing net (#0)
I0914 20:03:01.368085 4024 net.cpp:693] Ignoring source layer train_database
I0914 20:03:04.568979 4024 solver.cpp:404] Test net output #0: accuracy = 0.07575
I0914 20:03:04.569093 4024 solver.cpp:404] Test net output #1: loss = 2.20947 (* 1 = 2.20947 loss)
I0914 20:03:04.610549 4024 solver.cpp:228] Iteration 0, loss = 2.31814
I0914 20:03:04.610666 4024 solver.cpp:244] Train net output #0: loss = 2.31814 (* 1 = 2.31814 loss)
*** Aborted at 1473872584 (unix time) try "date -d @1473872584" if you are using GNU date ***
PC: @ 0x7f6870b62c52 caffe::SGDSolver<>::GetLearningRate()
*** SIGFPE (@0x7f6870b62c52) received by PID 4024 (TID 0x7f6871004a40) from PID 1890987090; stack trace: ***
@ 0x7f686f6bbcb0 (unknown)
@ 0x7f6870b62c52 caffe::SGDSolver<>::GetLearningRate()
@ 0x7f6870b62e44 caffe::SGDSolver<>::ApplyUpdate()
@ 0x7f6870b8e2fc caffe::Solver<>::Step()
@ 0x7f6870b8eb09 caffe::Solver<>::Solve()
@ 0x40821d train()
@ 0x40589c main
@ 0x7f686f6a6f45 (unknown)
@ 0x40610b (unknown)
@ 0x0 (unknown)
Floating point exception (core dumped)
我搜索了很多,main solutions that I've found是:
1.重新编译caffe文件。试过make clean
- &gt; make all
- &gt; make test
- &gt; make runtest
2.更改linux使用的驱动程序。我使用红色并更改为绿色(注意:我使用带有我的caffe的CPU,并且在makeconfig文件中提到了它):
所有这些都没有帮助,我仍然无法运行我的网络。
有没有人有想法?非常感谢,无论如何:)
这是完整的日志:
/home/roishik/anaconda2/bin/python /home/roishik/Desktop/Thesis/Code/cafe_cnn/third/code/run_network.py
I0914 20:03:01.142490 4024 caffe.cpp:210] Use CPU.
I0914 20:03:01.142940 4024 solver.cpp:48] Initializing solver from parameters:
test_iter: 400
test_interval: 400
base_lr: 0.001
display: 50
max_iter: 40000
lr_policy: "step"
gamma: 0.1
momentum: 0.9
weight_decay: 0.0005
snapshot: 5000
snapshot_prefix: "/home/roishik/Desktop/Thesis/Code/cafe_cnn/third/caffe_models/my_new/snapshots"
solver_mode: CPU
net: "/home/roishik/Desktop/Thesis/Code/cafe_cnn/third/caffe_models/my_new/fc_net_ver1.prototxt"
train_state {
level: 0
stage: ""
}
I0914 20:03:01.143082 4024 solver.cpp:91] Creating training net from net file: /home/roishik/Desktop/Thesis/Code/cafe_cnn/third/caffe_models/my_new/fc_net_ver1.prototxt
I0914 20:03:01.143712 4024 net.cpp:322] The NetState phase (0) differed from the phase (1) specified by a rule in layer validation_database
I0914 20:03:01.143754 4024 net.cpp:322] The NetState phase (0) differed from the phase (1) specified by a rule in layer accuracy
I0914 20:03:01.143913 4024 net.cpp:58] Initializing net from parameters:
name: "fc2Net"
state {
phase: TRAIN
level: 0
stage: ""
}
layer {
name: "train_database"
type: "Data"
top: "data"
top: "label"
include {
phase: TRAIN
}
transform_param {
mean_file: "/home/roishik/Desktop/Thesis/Code/cafe_cnn/third/input/mean.binaryproto"
}
data_param {
source: "/home/roishik/Desktop/Thesis/Code/cafe_cnn/third/input/train_lmdb"
batch_size: 200
backend: LMDB
}
}
layer {
name: "fc1"
type: "InnerProduct"
bottom: "data"
top: "fc1"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
inner_product_param {
num_output: 1024
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "relu1"
type: "ReLU"
bottom: "fc1"
top: "fc1"
}
layer {
name: "fc2"
type: "InnerProduct"
bottom: "fc1"
top: "fc2"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
inner_product_param {
num_output: 1024
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "relu2"
type: "ReLU"
bottom: "fc2"
top: "fc2"
}
layer {
name: "fc3"
type: "InnerProduct"
bottom: "fc2"
top: "fc3"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
inner_product_param {
num_output: 2
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "loss"
type: "SoftmaxWithLoss"
bottom: "fc3"
bottom: "label"
top: "loss"
}
I0914 20:03:01.144016 4024 layer_factory.hpp:77] Creating layer train_database
I0914 20:03:01.144811 4024 net.cpp:100] Creating Layer train_database
I0914 20:03:01.144846 4024 net.cpp:408] train_database -> data
I0914 20:03:01.144909 4024 net.cpp:408] train_database -> label
I0914 20:03:01.144951 4024 data_transformer.cpp:25] Loading mean file from: /home/roishik/Desktop/Thesis/Code/cafe_cnn/third/input/mean.binaryproto
I0914 20:03:01.153393 4035 db_lmdb.cpp:35] Opened lmdb /home/roishik/Desktop/Thesis/Code/cafe_cnn/third/input/train_lmdb
I0914 20:03:01.153481 4024 data_layer.cpp:41] output data size: 200,1,32,32
I0914 20:03:01.154615 4024 net.cpp:150] Setting up train_database
I0914 20:03:01.154670 4024 net.cpp:157] Top shape: 200 1 32 32 (204800)
I0914 20:03:01.154693 4024 net.cpp:157] Top shape: 200 (200)
I0914 20:03:01.154712 4024 net.cpp:165] Memory required for data: 820000
I0914 20:03:01.154742 4024 layer_factory.hpp:77] Creating layer fc1
I0914 20:03:01.154781 4024 net.cpp:100] Creating Layer fc1
I0914 20:03:01.154804 4024 net.cpp:434] fc1 <- data
I0914 20:03:01.154837 4024 net.cpp:408] fc1 -> fc1
I0914 20:03:01.159675 4036 blocking_queue.cpp:50] Waiting for data
I0914 20:03:01.215118 4024 net.cpp:150] Setting up fc1
I0914 20:03:01.215214 4024 net.cpp:157] Top shape: 200 1024 (204800)
I0914 20:03:01.215237 4024 net.cpp:165] Memory required for data: 1639200
I0914 20:03:01.215306 4024 layer_factory.hpp:77] Creating layer relu1
I0914 20:03:01.215342 4024 net.cpp:100] Creating Layer relu1
I0914 20:03:01.215363 4024 net.cpp:434] relu1 <- fc1
I0914 20:03:01.215387 4024 net.cpp:395] relu1 -> fc1 (in-place)
I0914 20:03:01.215417 4024 net.cpp:150] Setting up relu1
I0914 20:03:01.215440 4024 net.cpp:157] Top shape: 200 1024 (204800)
I0914 20:03:01.215459 4024 net.cpp:165] Memory required for data: 2458400
I0914 20:03:01.215478 4024 layer_factory.hpp:77] Creating layer fc2
I0914 20:03:01.215504 4024 net.cpp:100] Creating Layer fc2
I0914 20:03:01.215524 4024 net.cpp:434] fc2 <- fc1
I0914 20:03:01.215549 4024 net.cpp:408] fc2 -> fc2
I0914 20:03:01.264021 4024 net.cpp:150] Setting up fc2
I0914 20:03:01.264062 4024 net.cpp:157] Top shape: 200 1024 (204800)
I0914 20:03:01.264072 4024 net.cpp:165] Memory required for data: 3277600
I0914 20:03:01.264097 4024 layer_factory.hpp:77] Creating layer relu2
I0914 20:03:01.264118 4024 net.cpp:100] Creating Layer relu2
I0914 20:03:01.264129 4024 net.cpp:434] relu2 <- fc2
I0914 20:03:01.264143 4024 net.cpp:395] relu2 -> fc2 (in-place)
I0914 20:03:01.264166 4024 net.cpp:150] Setting up relu2
I0914 20:03:01.264181 4024 net.cpp:157] Top shape: 200 1024 (204800)
I0914 20:03:01.264190 4024 net.cpp:165] Memory required for data: 4096800
I0914 20:03:01.264201 4024 layer_factory.hpp:77] Creating layer fc3
I0914 20:03:01.264219 4024 net.cpp:100] Creating Layer fc3
I0914 20:03:01.264230 4024 net.cpp:434] fc3 <- fc2
I0914 20:03:01.264245 4024 net.cpp:408] fc3 -> fc3
I0914 20:03:01.264389 4024 net.cpp:150] Setting up fc3
I0914 20:03:01.264407 4024 net.cpp:157] Top shape: 200 2 (400)
I0914 20:03:01.264416 4024 net.cpp:165] Memory required for data: 4098400
I0914 20:03:01.264434 4024 layer_factory.hpp:77] Creating layer loss
I0914 20:03:01.264447 4024 net.cpp:100] Creating Layer loss
I0914 20:03:01.264459 4024 net.cpp:434] loss <- fc3
I0914 20:03:01.264469 4024 net.cpp:434] loss <- label
I0914 20:03:01.264487 4024 net.cpp:408] loss -> loss
I0914 20:03:01.264513 4024 layer_factory.hpp:77] Creating layer loss
I0914 20:03:01.264544 4024 net.cpp:150] Setting up loss
I0914 20:03:01.264559 4024 net.cpp:157] Top shape: (1)
I0914 20:03:01.264569 4024 net.cpp:160] with loss weight 1
I0914 20:03:01.264595 4024 net.cpp:165] Memory required for data: 4098404
I0914 20:03:01.264606 4024 net.cpp:226] loss needs backward computation.
I0914 20:03:01.264617 4024 net.cpp:226] fc3 needs backward computation.
I0914 20:03:01.264626 4024 net.cpp:226] relu2 needs backward computation.
I0914 20:03:01.264636 4024 net.cpp:226] fc2 needs backward computation.
I0914 20:03:01.264647 4024 net.cpp:226] relu1 needs backward computation.
I0914 20:03:01.264655 4024 net.cpp:226] fc1 needs backward computation.
I0914 20:03:01.264667 4024 net.cpp:228] train_database does not need backward computation.
I0914 20:03:01.264675 4024 net.cpp:270] This network produces output loss
I0914 20:03:01.264695 4024 net.cpp:283] Network initialization done.
I0914 20:03:01.265384 4024 solver.cpp:181] Creating test net (#0) specified by net file: /home/roishik/Desktop/Thesis/Code/cafe_cnn/third/caffe_models/my_new/fc_net_ver1.prototxt
I0914 20:03:01.265435 4024 net.cpp:322] The NetState phase (1) differed from the phase (0) specified by a rule in layer train_database
I0914 20:03:01.265606 4024 net.cpp:58] Initializing net from parameters:
name: "fc2Net"
state {
phase: TEST
}
layer {
name: "validation_database"
type: "Data"
top: "data"
top: "label"
include {
phase: TEST
}
transform_param {
mean_file: "/home/roishik/Desktop/Thesis/Code/cafe_cnn/second/input/mean.binaryproto"
}
data_param {
source: "/home/roishik/Desktop/Thesis/Code/cafe_cnn/second/input/validation_lmdb"
batch_size: 40
backend: LMDB
}
}
layer {
name: "fc1"
type: "InnerProduct"
bottom: "data"
top: "fc1"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
inner_product_param {
num_output: 1024
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "relu1"
type: "ReLU"
bottom: "fc1"
top: "fc1"
}
layer {
name: "fc2"
type: "InnerProduct"
bottom: "fc1"
top: "fc2"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
inner_product_param {
num_output: 1024
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "relu2"
type: "ReLU"
bottom: "fc2"
top: "fc2"
}
layer {
name: "fc3"
type: "InnerProduct"
bottom: "fc2"
top: "fc3"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
inner_product_param {
num_output: 2
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "accuracy"
type: "Accuracy"
bottom: "fc3"
bottom: "label"
top: "accuracy"
include {
phase: TEST
}
}
layer {
name: "loss"
type: "SoftmaxWithLoss"
bottom: "fc3"
bottom: "label"
top: "loss"
}
I0914 20:03:01.265750 4024 layer_factory.hpp:77] Creating layer validation_database
I0914 20:03:01.265878 4024 net.cpp:100] Creating Layer validation_database
I0914 20:03:01.265897 4024 net.cpp:408] validation_database -> data
I0914 20:03:01.265918 4024 net.cpp:408] validation_database -> label
I0914 20:03:01.265936 4024 data_transformer.cpp:25] Loading mean file from: /home/roishik/Desktop/Thesis/Code/cafe_cnn/second/input/mean.binaryproto
I0914 20:03:01.266034 4037 db_lmdb.cpp:35] Opened lmdb /home/roishik/Desktop/Thesis/Code/cafe_cnn/second/input/validation_lmdb
I0914 20:03:01.266098 4024 data_layer.cpp:41] output data size: 40,1,32,32
I0914 20:03:01.266295 4024 net.cpp:150] Setting up validation_database
I0914 20:03:01.266315 4024 net.cpp:157] Top shape: 40 1 32 32 (40960)
I0914 20:03:01.266330 4024 net.cpp:157] Top shape: 40 (40)
I0914 20:03:01.266340 4024 net.cpp:165] Memory required for data: 164000
I0914 20:03:01.266350 4024 layer_factory.hpp:77] Creating layer label_validation_database_1_split
I0914 20:03:01.266386 4024 net.cpp:100] Creating Layer label_validation_database_1_split
I0914 20:03:01.266404 4024 net.cpp:434] label_validation_database_1_split <- label
I0914 20:03:01.266422 4024 net.cpp:408] label_validation_database_1_split -> label_validation_database_1_split_0
I0914 20:03:01.266443 4024 net.cpp:408] label_validation_database_1_split -> label_validation_database_1_split_1
I0914 20:03:01.266464 4024 net.cpp:150] Setting up label_validation_database_1_split
I0914 20:03:01.266480 4024 net.cpp:157] Top shape: 40 (40)
I0914 20:03:01.266494 4024 net.cpp:157] Top shape: 40 (40)
I0914 20:03:01.266505 4024 net.cpp:165] Memory required for data: 164320
I0914 20:03:01.266515 4024 layer_factory.hpp:77] Creating layer fc1
I0914 20:03:01.266531 4024 net.cpp:100] Creating Layer fc1
I0914 20:03:01.266543 4024 net.cpp:434] fc1 <- data
I0914 20:03:01.266558 4024 net.cpp:408] fc1 -> fc1
I0914 20:03:01.320364 4024 net.cpp:150] Setting up fc1
I0914 20:03:01.320461 4024 net.cpp:157] Top shape: 40 1024 (40960)
I0914 20:03:01.320489 4024 net.cpp:165] Memory required for data: 328160
I0914 20:03:01.320533 4024 layer_factory.hpp:77] Creating layer relu1
I0914 20:03:01.320571 4024 net.cpp:100] Creating Layer relu1
I0914 20:03:01.320597 4024 net.cpp:434] relu1 <- fc1
I0914 20:03:01.320627 4024 net.cpp:395] relu1 -> fc1 (in-place)
I0914 20:03:01.320652 4024 net.cpp:150] Setting up relu1
I0914 20:03:01.320667 4024 net.cpp:157] Top shape: 40 1024 (40960)
I0914 20:03:01.320678 4024 net.cpp:165] Memory required for data: 492000
I0914 20:03:01.320689 4024 layer_factory.hpp:77] Creating layer fc2
I0914 20:03:01.320709 4024 net.cpp:100] Creating Layer fc2
I0914 20:03:01.320719 4024 net.cpp:434] fc2 <- fc1
I0914 20:03:01.320734 4024 net.cpp:408] fc2 -> fc2
I0914 20:03:01.361732 4024 net.cpp:150] Setting up fc2
I0914 20:03:01.361766 4024 net.cpp:157] Top shape: 40 1024 (40960)
I0914 20:03:01.361802 4024 net.cpp:165] Memory required for data: 655840
I0914 20:03:01.361821 4024 layer_factory.hpp:77] Creating layer relu2
I0914 20:03:01.361837 4024 net.cpp:100] Creating Layer relu2
I0914 20:03:01.361845 4024 net.cpp:434] relu2 <- fc2
I0914 20:03:01.361852 4024 net.cpp:395] relu2 -> fc2 (in-place)
I0914 20:03:01.361866 4024 net.cpp:150] Setting up relu2
I0914 20:03:01.361872 4024 net.cpp:157] Top shape: 40 1024 (40960)
I0914 20:03:01.361877 4024 net.cpp:165] Memory required for data: 819680
I0914 20:03:01.361881 4024 layer_factory.hpp:77] Creating layer fc3
I0914 20:03:01.361892 4024 net.cpp:100] Creating Layer fc3
I0914 20:03:01.361901 4024 net.cpp:434] fc3 <- fc2
I0914 20:03:01.361909 4024 net.cpp:408] fc3 -> fc3
I0914 20:03:01.362009 4024 net.cpp:150] Setting up fc3
I0914 20:03:01.362017 4024 net.cpp:157] Top shape: 40 2 (80)
I0914 20:03:01.362022 4024 net.cpp:165] Memory required for data: 820000
I0914 20:03:01.362032 4024 layer_factory.hpp:77] Creating layer fc3_fc3_0_split
I0914 20:03:01.362041 4024 net.cpp:100] Creating Layer fc3_fc3_0_split
I0914 20:03:01.362046 4024 net.cpp:434] fc3_fc3_0_split <- fc3
I0914 20:03:01.362053 4024 net.cpp:408] fc3_fc3_0_split -> fc3_fc3_0_split_0
I0914 20:03:01.362062 4024 net.cpp:408] fc3_fc3_0_split -> fc3_fc3_0_split_1
I0914 20:03:01.362073 4024 net.cpp:150] Setting up fc3_fc3_0_split
I0914 20:03:01.362082 4024 net.cpp:157] Top shape: 40 2 (80)
I0914 20:03:01.362088 4024 net.cpp:157] Top shape: 40 2 (80)
I0914 20:03:01.362093 4024 net.cpp:165] Memory required for data: 820640
I0914 20:03:01.362097 4024 layer_factory.hpp:77] Creating layer accuracy
I0914 20:03:01.362120 4024 net.cpp:100] Creating Layer accuracy
I0914 20:03:01.362128 4024 net.cpp:434] accuracy <- fc3_fc3_0_split_0
I0914 20:03:01.362134 4024 net.cpp:434] accuracy <- label_validation_database_1_split_0
I0914 20:03:01.362141 4024 net.cpp:408] accuracy -> accuracy
I0914 20:03:01.362152 4024 net.cpp:150] Setting up accuracy
I0914 20:03:01.362159 4024 net.cpp:157] Top shape: (1)
I0914 20:03:01.362164 4024 net.cpp:165] Memory required for data: 820644
I0914 20:03:01.362169 4024 layer_factory.hpp:77] Creating layer loss
I0914 20:03:01.362176 4024 net.cpp:100] Creating Layer loss
I0914 20:03:01.362181 4024 net.cpp:434] loss <- fc3_fc3_0_split_1
I0914 20:03:01.362187 4024 net.cpp:434] loss <- label_validation_database_1_split_1
I0914 20:03:01.362193 4024 net.cpp:408] loss -> loss
I0914 20:03:01.362226 4024 layer_factory.hpp:77] Creating layer loss
I0914 20:03:01.362251 4024 net.cpp:150] Setting up loss
I0914 20:03:01.362265 4024 net.cpp:157] Top shape: (1)
I0914 20:03:01.362277 4024 net.cpp:160] with loss weight 1
I0914 20:03:01.362298 4024 net.cpp:165] Memory required for data: 820648
I0914 20:03:01.362311 4024 net.cpp:226] loss needs backward computation.
I0914 20:03:01.362323 4024 net.cpp:228] accuracy does not need backward computation.
I0914 20:03:01.362336 4024 net.cpp:226] fc3_fc3_0_split needs backward computation.
I0914 20:03:01.362347 4024 net.cpp:226] fc3 needs backward computation.
I0914 20:03:01.362360 4024 net.cpp:226] relu2 needs backward computation.
I0914 20:03:01.362370 4024 net.cpp:226] fc2 needs backward computation.
I0914 20:03:01.362381 4024 net.cpp:226] relu1 needs backward computation.
I0914 20:03:01.362392 4024 net.cpp:226] fc1 needs backward computation.
I0914 20:03:01.362403 4024 net.cpp:228] label_validation_database_1_split does not need backward computation.
I0914 20:03:01.362416 4024 net.cpp:228] validation_database does not need backward computation.
I0914 20:03:01.362426 4024 net.cpp:270] This network produces output accuracy
I0914 20:03:01.362438 4024 net.cpp:270] This network produces output loss
I0914 20:03:01.362460 4024 net.cpp:283] Network initialization done.
I0914 20:03:01.362552 4024 solver.cpp:60] Solver scaffolding done.
I0914 20:03:01.362591 4024 caffe.cpp:251] Starting Optimization
I0914 20:03:01.362601 4024 solver.cpp:279] Solving fc2Net
I0914 20:03:01.362612 4024 solver.cpp:280] Learning Rate Policy: step
I0914 20:03:01.367985 4024 solver.cpp:337] Iteration 0, Testing net (#0)
I0914 20:03:01.368085 4024 net.cpp:693] Ignoring source layer train_database
I0914 20:03:04.568979 4024 solver.cpp:404] Test net output #0: accuracy = 0.07575
I0914 20:03:04.569093 4024 solver.cpp:404] Test net output #1: loss = 2.20947 (* 1 = 2.20947 loss)
I0914 20:03:04.610549 4024 solver.cpp:228] Iteration 0, loss = 2.31814
I0914 20:03:04.610666 4024 solver.cpp:244] Train net output #0: loss = 2.31814 (* 1 = 2.31814 loss)
*** Aborted at 1473872584 (unix time) try "date -d @1473872584" if you are using GNU date ***
PC: @ 0x7f6870b62c52 caffe::SGDSolver<>::GetLearningRate()
*** SIGFPE (@0x7f6870b62c52) received by PID 4024 (TID 0x7f6871004a40) from PID 1890987090; stack trace: ***
@ 0x7f686f6bbcb0 (unknown)
@ 0x7f6870b62c52 caffe::SGDSolver<>::GetLearningRate()
@ 0x7f6870b62e44 caffe::SGDSolver<>::ApplyUpdate()
@ 0x7f6870b8e2fc caffe::Solver<>::Step()
@ 0x7f6870b8eb09 caffe::Solver<>::Solve()
@ 0x40821d train()
@ 0x40589c main
@ 0x7f686f6a6f45 (unknown)
@ 0x40610b (unknown)
@ 0x0 (unknown)
Floating point exception (core dumped)
Done!
答案 0 :(得分:4)
查看您的错误消息:您收到SIGFPE
信号。这表明您有arithmetic error。此外,导致此错误的函数是评估学习速率的函数。
好像您没有在'solver.prototxt'