Question

我试图训练一个神经网络用于实值输出，我只是给出一组净插值点（看起来像方波振荡）但是反向传播总是不能给我一个好处适合输入，我试图添加更多的输入值更高的功能，并将输出标准化，但它似乎没有帮助。网络是3层1输入1隐藏1输出和一个输出节点我该如何解决这个问题？我也用这个成本函数是否正确？

for k = 1:m

  C= C+(y(k)-a2(k))^2;
end

我的代码：

clc;
clear all;
close all;
input_layer_size  = 4;
hidden_layer_size = 60;
num_labels = 1;
load('Xs');
load('Y-s');
theta1=randInitializeWeights(4, 60);
theta2=randInitializeWeights(60, 1);
plot (xq,vq)
hold on 
xq=polyFeatures(xq,4);
param=[theta1(:) ;theta2(:)];

[J ,Grad]= nnCostFunction(param,input_layer_size ,hidden_layer_size,num_labels,xq,vq,0);

       options = optimset('MaxIter', 50);
  costFunction = @(p) nnCostFunction(p, ...
                               input_layer_size, ...
                               hidden_layer_size, ...
                               num_labels, xq, vq, 10);


  [nn_params, cost] = fmincg(costFunction, param, options);

Theta1 = reshape(nn_params(1:hidden_layer_size * (input_layer_size + 1)), ...
             hidden_layer_size, (input_layer_size + 1));

Theta2 = reshape(nn_params((1 + (hidden_layer_size * (input_layer_size +   1))):end), ...
             num_labels, (hidden_layer_size + 1));

         l=xq(:,1);
         out =predictTest(Theta1,Theta2,xq);

          accuracy=mean(double(out == vq)) * 100
         plot (l,out,'yellow');

         hold off



    function [J grad] = nnCostFunction(nn_params, ...
        input_layer_size, ...
        hidden_layer_size, ...
        num_labels, ...
        X, y, lambda)

    y(841:901)=0;
    y=y/2.2;

    Theta1 = reshape( (nn_params(1:hidden_layer_size * (input_layer_size+1 ))), ...
        hidden_layer_size, (input_layer_size +1 ));

    Theta2 = reshape(nn_params((1+(hidden_layer_size * (input_layer_size +1))):end), ...
        num_labels, (hidden_layer_size +1 ));

    m = size(X, 1);
    J = 0;
    Theta1_grad = zeros(size(Theta1));
    Theta2_grad = zeros(size(Theta2));


    X= [ones(m,1) X];

    z1=X*Theta1';
    a1 = sigmoid(z1);
    a1= [ones(size(a1,1),1) a1];
    z2=a1*Theta2';
    a2= sigmoid(z2);


    for k = 1:m

        J= J+(y(k)-a2(k))^2;

    end
    J= J/m;
    Theta1(:,1)=zeros(1,size(Theta1,1));
    Theta2(:,1)=zeros(1,size(Theta2,1));
    s1=sum (sum (Theta1.^2));
    s2=sum (sum (Theta2.^2));

    s3= lambda *(s2 +s1 );
    s3=s3/(2*m);
    J=J+s3;

    D2=zeros(size(Theta2));
    D1=zeros(size(Theta1));
    for i= 1:m

        delta3=a2(i)-y(i);
        v=[0 sigmoidGradient(z1(i,:))];
        delta2=(Theta2'*delta3').*v';



        D2=D2+delta3'*a1(i,:) ;
        D1=D1+delta2(2:end)*X(i,:);


    end


    Theta1_grad = D1./m + (lambda/m)*[zeros(size(Theta1,1), 1) Theta1(:, 2:end)];
    Theta2_grad = D2./m + (lambda/m)*[zeros(size(Theta2,1), 1) Theta2(:, 2:end)];

    grad = [Theta1_grad(:) ; Theta2_grad(:)];


    end



    function W = randInitializeWeights(L_in, L_out)


    epsilon_init = 0.5;
    W = rand(L_out, 1 + L_in)*2*epsilon_init - epsilon_init;

    end

输入是1：9内插0.01增量，目标是0：2.2之间的数字，如方波脉冲

linear interpolation of data vs predicted in red

updated after increasing epochs

Answer 1

请注意，红线永远不会为零（最低约0.4 ），这表示训练有素的权重永远不会将网络输出归零（我的意思是权重需要为足够负，并且某些细胞中的偏差完全缺失或不是负的）

将信号从[-1扩展到1]并使用权重和偏差来训练网络以查看影响。将需要权重和偏见。
此处使用的简单神经网络不适合像方波那样的时间序列预测。使用here等预测模型作为时间序列

使用反向传播训练实值神经网络

1 个答案: