Question

我试图构建线性单层感知器（即没有隐藏层，所有输入连接到所有输出，线性激活功能）并使用delta规则训练它，一次一个数据点，但我没有得到我期待的结果。我使用均方误差作为我的损失函数，应该导致权重更新的衍生物只是learning_rate * error（* 2），但不知何故结果看起来与我的手动计算。我错过了什么？

import numpy as np
from keras.models import Sequential
from keras.optimizers import SGD
from keras.layers import Dense

features = np.array([[1,0,1],[0,1,1]])
features = np.tile(features, (500,1))
labels = np.array([[1,0],[0,1]])
labels = np.tile(labels, (500,1))

network = Sequential()
network.add(Dense(2, input_dim = 3, init = "zero", activation = "linear"))
network.compile(loss = "mse", optimizer = SGD(lr = 0.01))
network.fit(features, labels, nb_epoch = 1, batch_size = 1, shuffle = False)

network.get_weights()
# [[ 0.59687883, -0.39686254],
# [-0.39689422,  0.59687883],
# [ 0.19998412,  0.20001581]],

# manually
weights = np.array([[0.0,0.0],[0.0,0.0],[0.0,0.0]])
for i in range(500):
    summed_out1 = weights[0,0] + weights[2,0]
    summed_out2 = weights[0,1] + weights[2,1]
    change_out1 = 0.01 * (1.0 - summed_out1)
    change_out2 = 0.01 * (0.0 - summed_out2)
    weights[0,0] += change_out1
    weights[2,0] += change_out1
    weights[0,1] += change_out2
    weights[2,1] += change_out2
    #
    summed_out1 = weights[1,0] + weights[2,0]
    summed_out2 = weights[1,1] + weights[2,1]
    change_out1 = 0.01 * (0.0 - summed_out1)
    change_out2 = 0.01 * (1.0 - summed_out2)
    weights[1,0] += change_out1
    weights[2,0] += change_out1
    weights[1,1] += change_out2
    weights[2,1] += change_out2

weights
# [[ 0.66346388, -0.33011442],
# [-0.33014677,  0.66346388],
# [ 0.33331711,  0.33334946]]

Answer 1

我发现了问题。默认情况下，密集层包含偏差 - 一旦更改，网络就会显示所需的行为！

在keras中使用delta规则

1 个答案: