Question

我正在尝试构建一个具有3个输入，3个输出且没有较深层的简单神经网络，该网络将执行梯度下降以调整其权重，并能够将输入数组转换为输出数组。由于某种原因，它不起作用（预测会在几次迭代后停止更改），而且我似乎无法弄清楚原因。你能帮我吗？

当我有1个输入和3个输出，或3个输出和1个输入时，类似的代码起作用。我曾经像这样计算导数：derivatives = inputs * pure_error。但是现在我需要对权重矩阵进行此操作，我正在使用derivatives = np.outer(inputs, pure_errors)，这是我所做的唯一更改。

代码：

import numpy as np 

inputs = np.array([3, 5, 1])
weights = np.array([[0.1,0.2,0.05],
                    [1.0,0.53,0.5],
                    [1.7,2.3,1.2]])
target_predictions = np.array([1.5, 10, 93])
learning_rate = 0.05

def ann(inputs, weights):
    predictions = np.dot(weights, inputs)
    return predictions

# Initial Predictions
predictions = ann(inputs, weights)
errors = (predictions - target_predictions) ** 2
print("Predictions:",predictions, "Errors:",errors)

for i in range(350):
    predictions = ann(inputs, weights)
    errors = (predictions - target_predictions) ** 2

    print("Predictions:",predictions, "Error:",errors)

    pure_errors = predictions - target_predictions
    derivatives = np.outer(inputs, pure_errors)
    weight_updates = derivatives * learning_rate

    weights -= weight_updates

输出：

Predictions: [ 1.35  6.15 17.8 ] Error: [2.25000e-02 1.48225e+01 5.65504e+03]
Predictions: [15.585 29.875 22.545] Error: [ 198.387225  395.015625 4963.907025]
Predictions: [ 4.90875 12.08125 18.98625] Error: [1.16195766e+01 4.33160156e+00 5.47803519e+03]
.......
Predictions: [ 9.48428571 19.70714286 20.51142857] Error: [  63.74881837   94.22862245 5254.59298776]
Predictions: [ 9.48428571 19.70714286 20.51142857] Error: [  63.74881837   94.22862245 5254.59298776]
Predictions: [ 9.48428571 19.70714286 20.51142857] Error: [  63.74881837   94.22862245 5254.59298776]

Answer 1

发现错误。代替：

derivatives = np.outer(inputs, pure_errors)

我需要这样做：

derivatives = np.outer(pure_errors, inputs)

以错误的顺序传递变量会以某种方式交换结果并更新错误的权重。

简单的神经网络无法做出正确的预测。我在哪里弄错了？

1 个答案: