Question

I am attempting to make a neural network python module that can be imported form other programs. I am testing this module with some data from kaggle. However, no matter what the inputs are, the outputs is almost always the same, within 1e-7!! I have normalized my data and tried adding more hidden layers and more hidden nodes. I have also lowered my learning rate and made it lower as the training process continues. I tried adding a bias but that had a negative effect

Here is the importable hidden_net.py module:

class network:
    def __init__(self,layer_num,learning_rate=0.7,seed=None,logistic_coefficent=0.9):
        self.logistic_coefficent=logistic_coefficent
        self.learning_rate=learning_rate
        self.w0 = np.random.random((layer_num[0],layer_num[1]))
        self.w1 = np.random.random((layer_num[1],layer_num[2]))

        np.random.seed(seed)
    def sigmoid(self,x,reverse=False):
            if(reverse==True):
                return x*(1-x)
            return 1/(1+np.exp(-x))

    def train(self,inps,outs,mod_learn_rate=0):
        if mod_learn_rate == 0:
            mod_learn_=self.learning_rate
        inps=np.array(inps)
        layer0 = inps
        layer1 = self.sigmoid(np.dot(layer0,self.w0))
        layer2 = self.sigmoid(np.dot(layer1,self.w1))
        layer2_error = outs - layer2
        layer2_delta = layer2_error*self.sigmoid(layer2,reverse=True)#*mod_learn_rate
        layer1_error = layer2_delta.dot(self.w1.T)
        layer1_delta = layer1_error * self.sigmoid(layer1,reverse=True)#*mod_learn_rate

        self.w1 += layer2.T.dot(layer2_delta)
        self.w0 += layer1.T.dot(layer1_delta)
        return np.mean(abs(layer2_error))
    def calcout(self,inp):
        inp=np.array(inp)
        layer0=inp
        layer1=self.sigmoid(np.dot(layer0,self.w0))
        out=self.sigmoid(np.dot(layer1,self.w1))
        return out

And the script importing said module:

import random
from numpy import mean
random.seed(50404)
op=open('Mall_Customers_Mod.txt','r')
full=op.read()
op.close()
full_lines=full.split('\n')
training_lines=random.sample(full_lines,175)
training_inputs=[]
training_outputs=[]
for j in training_lines:
    training_inputs.append([float(j.split(',')[0]),float(j.split(',')[1]),float(j.split(',')[2])])
    training_outputs.append(float(j.split(',')[3]))
testing_lines=random.sample(full_lines,175)
testing_inputs=[]
testing_outputs=[]
for l in testing_lines:
    testing_inputs.append([float(l.split(',')[0]),float(l.split(',')[1]),float(j.split(',')[2])])
    testing_outputs.append(float(l.split(',')[3]))
nn=hidden_net.network([3,9,1],seed=50404,learning_rate=10)
er=[]
txt=''
try:
    for i in range(10000):
        for l in range(len(training_inputs)):
            er.append(nn.train(training_inputs[l],training_outputs[l],10/(i+1)))
        if (i%1000==0 or i==1 or i==0 or i==2):
            print('epoch:{}\nerror:{}\nlearning_rate={}'.format(i,mean(er),10/(i+1)))
            txt=txt+'\nepoch:{}\nerror:{}'.format(i,mean(er))
            er=[]
except KeyboardInterrupt:
    pass
print('done!')
score=0
error=[]
tests=0
for i in range(len(testing_inputs)):
    print('net output: ' +str(nn.calcout(testing_inputs[i])))
    print('true output: '+str(testing_outputs[i]))
    error.append(abs(nn.calcout(testing_inputs[i]) - testing_outputs[i]))
print('error: {}'.format(mean(error)))
print('\n\nweights:{}'.format(nn.w0))

The normalized data has the form

Sex     Age          Income  Spending Score
0,0.019230769230769232,0.0,0.3877551020408163
0,0.057692307692307696,0.0,0.8163265306122449
1,0.038461538461538464,0.00819672131147541,0.05102040816326531
1,0.09615384615384616,0.00819672131147541,0.7755102040816326
1,0.25,0.01639344262295082,0.3979591836734694

I would've expected the output to vary, but it doesn't

net output: [0.49777196]
true output: 0.3979591836734694
net output: [0.49999907]
true output: 0.8571428571428571
net output: [0.49997918]
true output: 0.7346938775510204
net output: [0.49983941]
true output: 0.30612244897959184
net output: [0.49999985]

This seems like a pretty common error with neural networks, with many causes. I would add a bounty but i can't afford one. Being trying different things for ages, I hope one of you can figure it out! Thanks in advance, 3NiGMa

Answer 1

您的学习率对于数据集的大小而言太高，请尝试将其提高到0.1或1，以便权重可以更快地更改

Neural Network returning the same output despite input change

1 个答案: