我正在尝试使用旧的backprop代码作为基础来实现rprop。我正在研究一个隐藏层的感知器。 Rprop算法相当简单，但我还没有想到所有的东西。这是我的代码：
for (j = 1; j <= nnh; j++)
{
network.input2[j] = network.w12[0][j];
for (i = 1; i <= nni; i++)
network.input2[j] += network.input[i] * network.w12[i][j];
network.output2[j] = (float)(1.0 / (1.0 + Math.Pow(Math.E, beta * -network.input2[j])));
}
for (k = 1; k <= nno; k++)
{
network.input3[k] = network.w23[0][k];
for (j = 1; j <= nnh; j++)
network.input3[k] += network.output2[j] * network.w23[j][k];
network.output[k] = (float)(1.0 / (1.0 + Math.Pow(Math.E, beta * -network.input3[k])));
error += (float)(0.5 * (t[k - 1] - network.output[k]) * (t[k - 1] - network.output[k]));
derivativeO[k] = (float)(t[k - 1] - network.output[k]) * network.output[k] * (1 - network.output[k]);
}
for (j = 1; j <= nnh; j++)
{
saw[j] = 0;
for (k = 1; k <= nno; k++)
saw[j] += derivativeO[k] * network.output2[j];
derivativeH[j] = saw[j] * network.output2[j] * (1 - network.output2[j]);
}
for (j = 1; j <= nnh; j++)//number of neurons in hidden layer
{
for (i = 1; i <= nni; i++)//number of inputs
{
network.gradientH[i][j] = network.input[i] * derivativeH[j];
if (network.gradientH[i][j] * network.gradientHPrev[i][j] > 0)
{
network.deltaH[i][j] = Math.Min(network.deltaH[i][j] * npos, dmax);
network.w12d[i][j] = -Math.Sign(network.gradientH[i][j]) * network.deltaH[i][j];
network.w12[i][j] += network.w12d[i][j];
network.gradientHPrev[i][j] = network.gradientH[i][j];
}
else if (network.gradientH[i][j] * network.gradientHPrev[i][j] < 0)
{
network.deltaH[i][j] = Math.Max(network.deltaH[i][j] * nneg, dmin);
network.gradientHPrev[i][j] = 0;
}
else if (network.gradientH[i][j] * network.gradientHPrev[i][j] == 0)
{
network.w12d[i][j] = -Math.Sign(network.gradientH[i][j]) * network.deltaH[i][j];
network.w12[i][j] += network.w12d[i][j];
network.gradientHPrev[i][j] = network.gradientH[i][j];
}
}
}
for (k = 1; k <= nno; k++)//number of outputs
{
for (j = 1; j <= nnh; j++)//number of neurons in hidden layer
{
network.gradientO[j][k] = network.output2[j] * derivativeO[k];
if (network.gradientOPrev[j][k] * network.gradientO[j][k] > 0)
{
network.deltaO[j][k] = Math.Min(network.deltaO[j][k] * npos, dmax);
network.w23d[j][k] = -Math.Sign(network.gradientO[j][k]) * network.deltaO[j][k];
network.w23[j][k] += network.w23d[j][k];
network.gradientOPrev[j][k] = network.gradientO[j][k];
}
else if (network.gradientOPrev[j][k] * network.gradientO[j][k] < 0)
{
network.deltaO[j][k] = Math.Max(network.deltaO[j][k] * nneg, dmin);
network.gradientOPrev[j][k] = 0;
}
else if (network.gradientOPrev[j][k] * network.gradientO[j][k] == 0)
{
network.w23d[j][k] = -Math.Sign(network.gradientO[j][k]) * network.deltaO[j][k];
network.w23[j][k] += network.w23d[j][k];
network.gradientOPrev[j][k] = network.gradientO[j][k];
}
}
}
前三个for循环与我在backprop中使用的相同。那部分代码运行正常。在重量更新期间出现问题。如果我正确地计算偏导数，我现在不行。网络有时会收敛，有时它只是随机行为。我认为其他一切都是正确的。有什么想法吗？
For循环从1开始，因为在之前的backprop实现中，偏差值存储在权重矩阵的第一个元素中。这是以前的backprop重量更新实现工作正常，也许它会使一些事情更清楚：
for (j = 1; j <= nnh; j++)
{
network.w12d[0][j] = learningRate * derivativeH[j] + momentum * network.w12d[0][j];
network.w12[0][j] += network.w12d[0][j];
for (i = 1; i <= nni; i++)
{
network.w12d[i][j] = learningRate * network.input[i] * derivativeH[j] + momentum * network.w12d[i][j];
network.w12[i][j] += network.w12d[i][j];
}
}
for (k = 1; k <= nno; k++)
{
network.w23d[0][k] = learningRate * derivativeO[k] + momentum * network.w23d[0][k];
network.w23[0][k] += network.w23d[0][k];
for (j = 1; j <= nnh; j++)
{
network.w23d[j][k] = learningRate * network.output2[j] * derivativeO[k] + momentum * network.w23d[j][k];
network.w23[j][k] += network.w23d[j][k];
}
}
答案 0 :(得分：0)
Encog RPROP实现有效。这是麻省理工学院的许可。看看他们在这里的实现进行比较：