Question

我目前正在构建机器-学习库。

问题的要点：当我向NN添加太多（也太大）的隐藏层时，它会输出NAN或-NAN。为什么会这样呢？我能做到这一点吗？如果我不能，图书馆可以/应该怎么做才能专业地处理它？</ p>

该库的（完整）单元测试之一如下所示：

struct testTrainWithMultipleLayers : public ZNN::NeuralNetwork<double>
{
    ZNN::NeuralNetwork<double> b;
    std::vector<std::vector<double>> input{{1, 0}, {0, 1}, {0, 0}, {1, 1}};
    std::vector<std::vector<double>> target{{1}, {1}, {0}, {0}};
    testTrainWithMultipleLayers()
    {
        withNeuralNetwork();
        trainForNIterations(1000);
        requireCorrectOutputs();
    }

    void withNeuralNetwork()
    {
        b.setInputLayerSize(2);
        b.addHiddenLayer(50);
        b.setOutputLayerSize(1);
        b.setLearningRate(0.7);
        b.setNormalization(ZNN::Fermi<double>());
    }

    void trainForNIterations(size_t iterations)
    {
         /////train the neural networks for so and so many iterations
    }

    void requireCorrectOutputs()
    {
        ///check the expected values were correctly approximated
    }
};

测试顺利通过！但是当我将withNeuralNetwork更改为此：

    void withNeuralNetwork()
    {
        b.setInputLayerSize(2);
        b.addHiddenLayer(50);
        b.addHiddenLayer(50);
        b.addHiddenLayer(50);
        b.addHiddenLayer(50);
        b.addHiddenLayer(50);
        b.addHiddenLayer(50);
        b.addHiddenLayer(50);
        b.addHiddenLayer(50);
        b.addHiddenLayer(50);
        b.addHiddenLayer(50);
        b.addHiddenLayer(50);
        b.setOutputLayerSize(1);
        b.setLearningRate(0.7);
        b.setNormalization(ZNN::Fermi<double>());
    }

我的单元测试的输出如下：

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test is a Catch v2.3.0 host application.
Run with -? for options

-------------------------------------------------------------------------------
Neural Network
-------------------------------------------------------------------------------
test_NN.hpp:9
...............................................................................

test_NN.hpp:263: FAILED:
  CHECK( Disabled by CATCH_CONFIG_DISABLE_STRINGIFICATION )
with expansion:
  Approx( nan ) == 2.0

test_NN.hpp:264: FAILED:
  CHECK( Disabled by CATCH_CONFIG_DISABLE_STRINGIFICATION )
with expansion:
  Approx( nan ) == 2.0

test_NN.hpp:265: FAILED:
  CHECK( Disabled by CATCH_CONFIG_DISABLE_STRINGIFICATION )
with expansion:
  Approx( nan ) == 1.0

test_NN.hpp:266: FAILED:
  CHECK( Disabled by CATCH_CONFIG_DISABLE_STRINGIFICATION )
with expansion:
  Approx( nan ) == 1.0

===============================================================================
test cases:  5 |  4 passed | 1 failed
assertions: 50 | 46 passed | 4 failed

NN输出NAN或-NAN（取决于设置）

可能值得注意的是，这不仅发生在我的特定单元测试中，而且还发生在我使用该库创建的每个NN应用程序中（到目前为止，这在我看来并没有什么大不了的缩小NN的大小直到起作用为止。

我已经找到了第一次发生这种情况的地方：它发生在改变Neuron权重的函数中，算法得到的导数为nan，从而使整个NN级联为输出NAN。

我确保不进行0除。整个NN代码中唯一的除法是用于计算NN的误差的1/2 * error和归一化中的除法，但我确保那里也不除以0。或者更确切地说：求解exp (x) = 0时，x是未定义的

单元测试的代码在这里：https://github.com/Wittmaxi/ZENeural/blob/master/library/tests/test_NN.cpp#L220

修改

当我将withNeuralNetwork更改为此时，也会发生这种情况：

    void withNeuralNetwork()
    {
        b.setInputLayerSize(2);
        b.addHiddenLayer(5000);
        b.setOutputLayerSize(1);
        b.setLearningRate(0.7);
        b.setNormalization(ZNN::Fermi<double>());
    }

当隐藏层过多时，神经网络会输出nan

0 个答案: