Question

也许有人可以帮我。我有点卡住。现在，我正在尝试用C＃编写自己的神经网络。我得到了一些工作（它与XOR一起工作）。这是一个具有输入，隐藏和输出的简单神经网络，我将ReLU用作激活函数。我的问题是，当我将“隐藏层”的数量增加到大于〜16时，我倾向于获得一些NaN或Infinites，这会很快使所有内容混乱。我试图降低学习率，但这无济于事。我认为问题出在我的SGD函数中，但是我找不到真正的问题，尤其是因为它工作在更少的层上。

这是功能：

private void SGD(double learningRate, double[] weightedSumHidden, double[] errors_output)
    {
        /*---------------------------------------------------------------
        * -- Calculate Delta of the weight between hidden and output --
        ---------------------------------------------------------------*/
        var HiddenTransposed = Hidden.Transpose();
        var deltaWeightOutput = HiddenTransposed.Dot(errors_output);
        double[,] deltaWeightOutput2D = Matrix.Create(deltaWeightOutput); //Convert to Matrix
        WeightsHiddenOutput = WeightsHiddenOutput.Add(deltaWeightOutput2D.Multiply(learningRate));

        /*---------------------------------------------------------------
         * -- Calculate Delta of the weight between input and hidden --
         ---------------------------------------------------------------*/
        //First we have to calculate the Error in the hidden nodes ...
        //Transposed because we are going Backwards through the Network
        var WHOTransposed = WeightsHiddenOutput.Transpose();
        //Moves the Error to the output layer
        var errors_hidden = WHOTransposed.Dot(errors_output);
        //Element Wise multiplication (schur product)
        weightedSumHidden = ApplyDerivativeReLU(weightedSumHidden);
        //Moves the Error backthrough the Neuron
        errors_hidden = errors_hidden.Multiply(weightedSumHidden);

        //... then we can Calculate the Delta
        var InputTransposed = Inputs.Transpose();
        var deltaWeightHidden = InputTransposed.Dot(errors_hidden);
        double[,] deltaWeightHidden2D = Matrix.Create(deltaWeightHidden); //Convert to Matrix
        deltaWeightHidden2D = Inputs.Transpose().Dot(deltaWeightHidden2D);

        /*---------------------------------------------------------------
         * --        Adjust Weights and Biases using the delta         --
         ---------------------------------------------------------------*/
        //The Biases just get adjusted by adding the Errors multiplied by the learning rate
        BiasOutput = BiasOutput.Add(errors_output.Multiply(learningRate)); //Output Bias
        BiasHidden = BiasHidden.Add(errors_hidden.Multiply(learningRate)); //Hidden Bias

        WeightsInputHidden = WeightsInputHidden.Add(deltaWeightHidden2D.Multiply(learningRate));           
    }

如果有人可以帮助我，我将非常感激我坚持了好几天。我使用本指南（http://neuralnetworksanddeeplearning.com/chap2.html）作为代码的基础。另外，我在矩阵数学中使用Accord.Math。

谢谢！

Answer 1

您可以将它们与断点一起使用，以检查错误的开始位置：

if (double.IsNan(value))
if (double.IsInfinity(value))
if (float.IsNan(value))
if (float.IsInfinity(value))

我在使用NaN时遇到了同样的问题，而异常帮助我找到了问题：

if (double.IsNan(value) || double.IsIninity(value)) throw new Exception();

Visual Studio的调试工具非常有用-您可以使用断点来检查对象中的值。

C＃中的神经网络-NaN和无限

1 个答案: