Question

我正在尝试让MLP工作。我的目标是在给定Yt，Yt-1 ...，Yt-2时让网络预测输出Yt-10。我一直在使用生成的数据集，这应该没问题。我的网将始终输出一条直线，并将上下移动该线。这条线在任何地方都是平坦的，它不会开始弯曲。我的目标是sin(x)曲线的一部分。

这是我的代码：

数据集：

% Underlying function
syms f(a)
f(a)=sin(a/50);

% Data: lagged
x=[1:100+10]';
% add noise
y=eval(f(x))+0.01*randn(numel(x),1); 

% rescale dataset to [0,1]
y=(y-min(y))/(max(y)-min(y));

% Lagged predictor
X=[];
for L=1:10
    temp=lagmatrix(y,-L);
    X=horzcat(X,temp);
end

% train set size m
y_training=y(1:100);
X_training=X(1:100,:);

训练：

%% Network

% Initialize weights
W1=randn(10,10)/10; Wb1=randn(10,1)/10;
W2=randn(10,10)/10; Wb2=randn(10,1)/10;
W3=randn(1,10)/10;  Wb3=randn(1,1)/10;

% Activation function
syms f(z); 
f(z)=1/(1+exp(-z));
Df=diff(f);

% Net parameters
alpha=3;
lambda=0;

costs=[];
m=numel(y_training); % size of training set
no_iter=500; % number of iterations of grad descent 
p=1;


for j=1:no_iter

    %Initialize error
    deltaW1=zeros(size(W1));  deltaWb1=zeros(size(Wb1));
    deltaW2=zeros(size(W2));  deltaWb2=zeros(size(Wb2));
    deltaW3=zeros(size(W3));  deltaWb3=zeros(size(Wb3));

    JW=0;
    keep_output=[];

    for i=1:m

        % input layer
        a1=[X_training(i,:)']; y_true=y_training(i);

        % FP activations
        z2=[W1 Wb1]*[a1; 1]; a2=eval(f(z2));
        z3=[W2 Wb2]*[a2; 1]; a3=eval(f(z3));
        z4=[W3 Wb3]*[a3; 1]; a4=eval(f(z4)); 

        % BP individual errors 
        delta_a4= -(y_true-a4) .* eval(Df(z4));
        delta_a3= W3'*delta_a4 .* eval(Df(z3));
        delta_a2= W2'*delta_a3 .* eval(Df(z2));

        % DJDW for each parameter
        DJDW3=delta_a4*(a3)'; DJDWb3=delta_a4;
        DJDW2=delta_a3*(a2)'; DJDWb2=delta_a3; 
        DJDW1=delta_a2*(a1)'; DJDWb1=delta_a2;     

        % summing DJDW of each example
        deltaW1=deltaW1+DJDW1; deltaWb1=deltaWb1+DJDWb1;
        deltaW2=deltaW2+DJDW2; deltaWb2=deltaWb2+DJDWb2;
        deltaW3=deltaW3+DJDW3; deltaWb3=deltaWb3+DJDWb3;

        % Cost function
        JW_xy=(y_true-a4)^2; % single example
        JW=JW+JW_xy; % aggregate

        % store output and true values
        keep_output=[keep_output a4];

    end

    % update weights according to average cost of current iteration
    W1=W1-alpha*( (1/m)*deltaW1) ; Wb1=Wb1-alpha*( (1/m)*deltaWb1);
    W2=W2-alpha*( (1/m)*deltaW2) ; Wb2=Wb2-alpha*( (1/m)*deltaWb2);
    W3=W3-alpha*( (1/m)*deltaW3) ; Wb3=Wb3-alpha*( (1/m)*deltaWb3);   

    clf, plot(y_training), hold on
    plot(keep_output);
    drawnow

end

Answer 1

似乎只有反向传播算法不起作用。我观察到的是每次迭代时上下移动的线。问题是我没有缩放输出。

鉴于我缩放了输入数据集，输出也应该缩放。

如何让反向传播为MLP工作？ MATLAB

1 个答案: