Question

我正在建立一个神经网络。所有对的训练输出为0或1。我注意到，如果仅添加目标输出为“ 1”的单个训练对，而其他9对为“ 0”，则训练后我的权重全部为负，但是如果我增加了训练集中“ 1”目标输出的数量，我也看到了积极的权重。

给出所有负权重的训练集：输入：

[[0.46       0.4        0.98039216]
 [0.58       0.         0.98039216]
 [0.2        1.         0.39215686]
 [0.1        0.4        0.45960784]
 [0.74       0.53333333 0.19607843]
 [0.48       0.93333333 0.        ]
 [0.38       0.7        0.98039216]
 [0.02       0.53333333 1.        ]
 [0.         0.03333333 0.88235294]
 [1.         0.8        0.78431373]]

输出：

[[0.][0.][0.][0.][0.][0.][0.][0.][0.][1.]]

训练前的体重（随机）：

[[-0.16595599]
 [ 0.44064899]
 [-0.99977125]]

训练后的体重：

[[-1.48868116]
 [-4.8662876 ]
 [-5.42639621]]

但是，如果我这样将目标输出再增加一个“ 1”

[[0.][0.][0.][0.][0.][0.][0.][0.][0.][1.]]

训练后我的体重也随之增加：

[[ 1.85020129]
 [-1.9759502 ]
 [-1.03829837]]

这可能是什么原因？训练时可能是太多的“ 0”使“ 1”微不足道吗？如果是这样，训练时我应该如何改变方法？我想将此训练与一个训练集一起使用，该训练集包含大约480个训练对，输出为“ 0”，而输出20个为“ 1”

（我正在使用S型函数：）

完整代码：

from numpy import exp, array, random, dot
from collections import defaultdict
import csv
import numpy as np

class NeuralNetwork():
    def __init__(self):

        random.seed(1)

        self.synaptic_weights = 2 * random.random((3, 1)) - 1

    def __sigmoid(self, x):
        return 1 / (1 + exp(-x))

    def __sigmoid_derivative(self, x):
        return x * (1 - x)

    def train(self, training_set_inputs, training_set_outputs, number_of_training_iterations):
        for iteration in range(number_of_training_iterations):

            output = self.think(training_set_inputs)
            error = training_set_outputs - output
            adjustment = training_set_inputs.T.dot(error * self.__sigmoid_derivative(output))

            self.synaptic_weights += adjustment


    def think(self, inputs):
        return self.__sigmoid(dot(inputs, self.synaptic_weights))


if __name__ == "__main__":

    neural_network = NeuralNetwork()

    print ("Random starting synaptic weights: ")
    print (neural_network.synaptic_weights)

    training_set_inputs = array([
     [0.46,0.4,0.98039216],
     [0.58,0.0,0.98039216],
     [0.2,1.0,0.39215686],
     [0.1,0.4,0.45960784],
     [0.74,0.53333333,0.19607843],
     [0.48,0.93333333,0.0],
     [0.38,0.7,0.98039216],
     [0.02,0.53333333,1.0],
     [0.,0.03333333,0.88235294],
     [1.0,0.8,0.78431373]])

    training_set_outputs = array([[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0]]).T

    neural_network.train(training_set_inputs, training_set_outputs, 10000)

    print ("New synaptic weights after training: ")
    print (neural_network.synaptic_weights)

    print ("Considering new situation [0.5,0.5,0.5] -> ?: ")
    test = [0.5,0.5,0.5]
    print (neural_network.think(array(test)))

有什么想法吗？

谢谢

Answer 1

在计算更新时，您似乎忘记了学习率。尝试将“调整”乘以0.001

        error = training_set_outputs - output
        adjustment = training_set_inputs.T.dot(error * self.__sigmoid_derivative(output))

        self.synaptic_weights += adjustment

训练后所有的体重都变为负数

1 个答案: