我使用指数衰减来在每10个时期之后衰减我的学习率。正如你在下面的输出中看到的那样，当我的学习率开始下降时，我的损失开始增加，我尝试了几种变化，每次都会发生同样的事情。可能出现什么问题？
global_step = tf.Variable(0, name="global_step", trainable=False)
decayed_learning_rate = tf.train.exponential_decay(learning_rate = 0.0001,global_step = global_step,decay_steps = 1000, decay_rate = 0.6, staircase = True)
optimizer= tf.train.MomentumOptimizer(learning_rate = decayed_learning_rate, momentum = 0.9)
minimize_loss = optimizer.minimize(loss, global_step=global_step)
这是输出：
1
Epoch Finished
Loss after one Epoch(Training) = 8.291080, Training Accuracy= 0.18000
Loss after one Epoch(Validation) = 8.464677, Validation Accuracy= 0.10000
Loss after one Epoch(Test) = 8.631430, Test Accuracy= 0.13000
2
Epoch Finished
Loss after one Epoch(Training) = 4.619487, Training Accuracy= 0.12000
Loss after one Epoch(Validation) = 4.835144, Validation Accuracy= 0.14000
Loss after one Epoch(Test) = 5.233496, Test Accuracy= 0.09000
3
Epoch Finished
Loss after one Epoch(Training) = 4.591153, Training Accuracy= 0.10000
Loss after one Epoch(Validation) = 4.878084, Validation Accuracy= 0.09000
Loss after one Epoch(Test) = 4.112285, Test Accuracy= 0.11000
4
Epoch Finished
Loss after one Epoch(Training) = 4.530641, Training Accuracy= 0.13000
Loss after one Epoch(Validation) = 4.874103, Validation Accuracy= 0.10000
Loss after one Epoch(Test) = 4.225502, Test Accuracy= 0.14000
5
Epoch Finished
Loss after one Epoch(Training) = 3.664831, Training Accuracy= 0.26000
Loss after one Epoch(Validation) = 3.207108, Validation Accuracy= 0.29000
Loss after one Epoch(Test) = 4.435939, Test Accuracy= 0.17000
6
Epoch Finished
Loss after one Epoch(Training) = 3.682740, Training Accuracy= 0.26000
Loss after one Epoch(Validation) = 3.794605, Validation Accuracy= 0.21000
Loss after one Epoch(Test) = 3.890673, Test Accuracy= 0.17000
7
Epoch Finished
Loss after one Epoch(Training) = 3.638363, Training Accuracy= 0.27000
Loss after one Epoch(Validation) = 4.057161, Validation Accuracy= 0.21000
Loss after one Epoch(Test) = 4.400304, Test Accuracy= 0.19000
8
Epoch Finished
Loss after one Epoch(Training) = 3.290856, Training Accuracy= 0.13000
Loss after one Epoch(Validation) = 3.573865, Validation Accuracy= 0.02000
Loss after one Epoch(Test) = 3.289892, Test Accuracy= 0.13000
9
Epoch Finished
Loss after one Epoch(Training) = 3.249848, Training Accuracy= 0.13000
Loss after one Epoch(Validation) = 3.816904, Validation Accuracy= 0.09000
Loss after one Epoch(Test) = 3.365518, Test Accuracy= 0.09000
10
Epoch Finished
Loss after one Epoch(Training) = 3.261417, Training Accuracy= 0.13000
Loss after one Epoch(Validation) = 3.051553, Validation Accuracy= 0.13000
Loss after one Epoch(Test) = 3.935049, Test Accuracy= 0.10000
11
Epoch Finished
Loss after one Epoch(Training) = 3.274293, Training Accuracy= 0.12000
Loss after one Epoch(Validation) = 3.341079, Validation Accuracy= 0.12000
Loss after one Epoch(Test) = 3.465601, Test Accuracy= 0.09000
12
Epoch Finished
Loss after one Epoch(Training) = 3.245074, Training Accuracy= 0.12000
Loss after one Epoch(Validation) = 3.655849, Validation Accuracy= 0.09000
Loss after one Epoch(Test) = 3.890745, Test Accuracy= 0.11000
13
Epoch Finished
Loss after one Epoch(Training) = 3.242341, Training Accuracy= 0.12000
Loss after one Epoch(Validation) = 3.527991, Validation Accuracy= 0.04000
Loss after one Epoch(Test) = 3.207819, Test Accuracy= 0.12000
14
Epoch Finished
Loss after one Epoch(Training) = 3.277830, Training Accuracy= 0.12000
Loss after one Epoch(Validation) = 3.797029, Validation Accuracy= 0.10000
Loss after one Epoch(Test) = 3.317770, Test Accuracy= 0.11000
15
Epoch Finished
Loss after one Epoch(Training) = 3.269509, Training Accuracy= 0.12000
Loss after one Epoch(Validation) = 3.074466, Validation Accuracy= 0.12000
Loss after one Epoch(Test) = 3.887167, Test Accuracy= 0.10000
16
Epoch Finished
Loss after one Epoch(Training) = 4.100363, Training Accuracy= 0.14000
Loss after one Epoch(Validation) = 4.208894, Validation Accuracy= 0.10000
Loss after one Epoch(Test) = 4.150678, Test Accuracy= 0.15000
17
Epoch Finished
Loss after one Epoch(Training) = 4.037428, Training Accuracy= 0.14000
Loss after one Epoch(Validation) = 4.366947, Validation Accuracy= 0.10000
Loss after one Epoch(Test) = 4.501517, Test Accuracy= 0.09000
18
Epoch Finished
Loss after one Epoch(Training) = 4.048151, Training Accuracy= 0.14000
Loss after one Epoch(Validation) = 4.315053, Validation Accuracy= 0.10000
Loss after one Epoch(Test) = 3.972508, Test Accuracy= 0.10000
19
Epoch Finished
Loss after one Epoch(Training) = 4.046428, Training Accuracy= 0.14000
Loss after one Epoch(Validation) = 4.649216, Validation Accuracy= 0.08000
Loss after one Epoch(Test) = 4.125694, Test Accuracy= 0.11000
20
Epoch Finished
Loss after one Epoch(Training) = 4.082591, Training Accuracy= 0.13000
Loss after one Epoch(Validation) = 3.639134, Validation Accuracy= 0.16000
Loss after one Epoch(Test) = 4.476624, Test Accuracy= 0.16000
21
Epoch Finished
Loss after one Epoch(Training) = 4.068653, Training Accuracy= 0.13000
Loss after one Epoch(Validation) = 4.141028, Validation Accuracy= 0.10000
Loss after one Epoch(Test) = 4.086758, Test Accuracy= 0.15000
22
Epoch Finished
Loss after one Epoch(Training) = 4.066084, Training Accuracy= 0.13000
Loss after one Epoch(Validation) = 4.252730, Validation Accuracy= 0.10000
Loss after one Epoch(Test) = 4.357038, Test Accuracy= 0.10000
23
Epoch Finished
Loss after one Epoch(Training) = 4.031103, Training Accuracy= 0.14000
Loss after one Epoch(Validation) = 4.360917, Validation Accuracy= 0.10000
Loss after one Epoch(Test) = 3.916987, Test Accuracy= 0.11000
24
Epoch Finished
Loss after one Epoch(Training) = 4.031075, Training Accuracy= 0.14000
Loss after one Epoch(Validation) = 4.653004, Validation Accuracy= 0.07000
Loss after one Epoch(Test) = 4.183711, Test Accuracy= 0.10000
25
Epoch Finished
Loss after one Epoch(Training) = 4.039016, Training Accuracy= 0.14000
Loss after one Epoch(Validation) = 3.654388, Validation Accuracy= 0.15000
Loss after one Epoch(Test) = 4.228384, Test Accuracy= 0.18000