我尝试使用梯度下降反向传播在Python中重建一个简单的MLP人工神经网络。我的目标是尝试重新创建MATLAB的人工神经网络所产生的准确性，但我甚至没有接近。我使用与MATLAB相同的参数;相同数量的隐藏节点（20），1000个纪元，0.01的学习速率（alpha）和相同的数据（显然），但我的代码在改进结果方面没有任何进展，而MATLAB在98％的范围内获得了准确性。 / p>
我试图通过MATLAB进行调试，看看它在做什么，但我没有太多运气。我相信MATLAB将输入数据在0和1之间进行缩放，并为输入添加偏差，这些都是我在Python代码中使用的。
MATLAB做的是什么，产生的结果要高得多？或者，更可能的是，我在Python代码中做错了什么导致产生如此糟糕的结果？我能想到的只是重量的不良启动，数据读取不正确，或者处理数据的操作不正确，或者激活函数不正确/不良（我也尝试过tanh，结果相同）。< / p>
我的尝试如下，基于我在网上找到的代码并略微调整以读取我的数据，而MATLAB脚本（仅11行代码）低于此。在底部是我使用的数据集的链接（我也通过MATLAB获得）：
感谢您的帮助。
Main.py
import numpy as np
import Process
import matplotlib.pyplot as plt
from sklearn.metrics import confusion_matrix, classification_report
from sklearn.cross_validation import train_test_split
from sklearn.preprocessing import LabelBinarizer
import warnings
def sigmoid(x):
return 1.0/(1.0 + np.exp(-x))
def sigmoid_prime(x):
return sigmoid(x)*(1.0-sigmoid(x))
class NeuralNetwork:
def __init__(self, layers):
self.activation = sigmoid
self.activation_prime = sigmoid_prime
# Set weights
self.weights = []
# layers = [2,2,1]
# range of weight values (-1,1)
# input and hidden layers - random((2+1, 2+1)) : 3 x 3
for i in range(1, len(layers) - 1):
r = 2*np.random.random((layers[i-1] + 1, layers[i] + 1)) - 1
self.weights.append(r)
# output layer - random((2+1, 1)) : 3 x 1
r = 2*np.random.random((layers[i] + 1, layers[i+1])) - 1
self.weights.append(r)
def fit(self, X, y, learning_rate, epochs):
# Add column of ones to X
# This is to add the bias unit to the input layer
ones = np.atleast_2d(np.ones(X.shape[0]))
X = np.concatenate((ones.T, X), axis=1)
for k in range(epochs):
i = np.random.randint(X.shape[0])
a = [X[i]]
for l in range(len(self.weights)):
dot_value = np.dot(a[l], self.weights[l])
activation = self.activation(dot_value)
a.append(activation)
# output layer
error = y[i] - a[-1]
deltas = [error * self.activation_prime(a[-1])]
# we need to begin at the second to last layer
# (a layer before the output layer)
for l in range(len(a) - 2, 0, -1):
deltas.append(deltas[-1].dot(self.weights[l].T)*self.activation_prime(a[l]))
# reverse
# [level3(output)->level2(hidden)] => [level2(hidden)->level3(output)]
deltas.reverse()
# backpropagation
# 1. Multiply its output delta and input activation
# to get the gradient of the weight.
# 2. Subtract a ratio (percentage) of the gradient from the weight.
for i in range(len(self.weights)):
layer = np.atleast_2d(a[i])
delta = np.atleast_2d(deltas[i])
self.weights[i] += learning_rate * layer.T.dot(delta)
def predict(self, x):
a = np.concatenate((np.ones(1).T, np.array(x)))
for l in range(0, len(self.weights)):
a = self.activation(np.dot(a, self.weights[l]))
return a
# Create neural net, 13 inputs, 20 hidden nodes, 3 outputs
nn = NeuralNetwork([13, 20, 3])
data = Process.readdata('wine')
# Split data out into input and output
X = data[0]
y = data[1]
# Normalise input data between 0 and 1.
X -= X.min()
X /= X.max()
# Split data into training and test sets (15% testing)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.15)
# Create binay output form
y_ = LabelBinarizer().fit_transform(y_train)
# Train data
lrate = 0.01
epoch = 1000
nn.fit(X_train, y_, lrate, epoch)
# Test data
err = []
for e in X_test:
# Create array of output data (argmax to get classification)
err.append(np.argmax(nn.predict(e)))
# Hide warnings. UndefinedMetricWarning thrown when confusion matrix returns 0 in any one of the classifiers.
warnings.filterwarnings('ignore')
# Produce confusion matrix and classification report
print(confusion_matrix(y_test, err))
print(classification_report(y_test, err))
# Plot actual and predicted data
plt.figure(figsize=(10, 8))
target, = plt.plot(y_test, color='b', linestyle='-', lw=1, label='Target')
estimated, = plt.plot(err, color='r', linestyle='--', lw=3, label='Estimated')
plt.legend(handles=[target, estimated])
plt.xlabel('# Samples')
plt.ylabel('Classification Value')
plt.grid()
plt.show()
Process.py
import csv
import numpy as np
# Add constant column of 1's
def addones(arrayvar):
return np.hstack((np.ones((arrayvar.shape[0], 1)), arrayvar))
def readdata(loc):
# Open file and calculate the number of columns and the number of rows. The number of rows has a +1 as the 'next'
# operator in num_cols has already pasted over the first row.
with open(loc + '.input.csv') as f:
file = csv.reader(f, delimiter=',', skipinitialspace=True)
num_cols = len(next(file))
num_rows = len(list(file))+1
# Create a zero'd array based on the number of column and rows previously found.
x = np.zeros((num_rows, num_cols))
y = np.zeros(num_rows)
# INPUT #
# Loop through the input file and put each row into a new row of 'samples'
with open(loc + '.input.csv', newline='') as csvfile:
file = csv.reader(csvfile, delimiter=',')
count = 0
for row in file:
x[count] = row
count += 1
# OUTPUT #
# Do the same and loop through the output file.
with open(loc + '.output.csv', newline='') as csvfile:
file = csv.reader(csvfile, delimiter=',')
count = 0
for row in file:
y[count] = row[0]
count += 1
# Set data type
x = np.array(x).astype(np.float)
y = np.array(y).astype(np.int)
return x, y
MATLAB脚本
%% LOAD DATA
[x1,t1] = wine_dataset;
%% SET UP NN
net = patternnet(20);
net.trainFcn = 'traingd';
net.layers{2}.transferFcn = 'logsig';
net.derivFcn = 'logsig';
%% TRAIN AND TEST
[net,tr] = train(net,x1,t1);
答案 0 :(得分：0)
我认为您混淆了epoch和step这两个词。如果您已经培训了一个epoch，那么它通常指的是已经遍历了所有数据。
例如：如果您有10,000个样本，那么您已经通过模型放置了所有10.000个样本（无视样本的随机抽样），并且每次都采取了一个步骤（更新了权重）。
修复：运行您的网络的时间更长：
nn.fit(X_train, y_, lrate, epoch*len(X))
<强>加成：
MatLab的文档会将时期转换为(iterations) here，这会产生误导，但会对其进行评论here，这基本上就是我上面写的内容。
答案 1 :(得分：0)
我相信我发现了这个问题。这是数据集本身的组合（所有数据集都没有出现此问题）以及我对数据进行缩放的方式。我原来的缩放方法，处理0到1之间的结果，没有帮助这种情况，并导致看到不好的结果：
# Normalise input data between 0 and 1.
X -= X.min()
X /= X.max()
我找到了另一种缩放方法，由sklearn预处理包提供：
from sklearn import preprocessing
X = preprocessing.scale(X)
这种缩放方法不在0和1之间，我有进一步的调查来确定它为什么有这么多帮助，但结果现在回来的准确率为96％到100％。与MATLAB结果非常相似，我认为它使用的是类似（或相同）的预处理缩放方法。
正如我上面所说，并非所有数据集都是如此。使用内置的sklearn虹膜或数字数据集似乎可以在不缩放的情况下产生良好的效果。