Question

我有一个专家混合代码，可以使用少量参数。它如下：

global x_au;
global x_vi;
global x_alpha;
global y;
global parameter;

options = optimoptions(@fminunc,'GradObj', 'on', 'Algorithm','quasi-newton','MaxIter', 10000,'Display','iter-detailed'); % change number of iterations
optTheta=[];

x_au=x_au_train;
x_vi=x_vi_train;
x_alpha=x_alpha_train;
y=y_train;
parameter=zeros(8969,1);

%expectation step
fprintf('opt1 begins');
opt_1;
fprintf('opt1 complete');
%maximaization step
[x] = fminunc(@costfunction,parameter(1:4483),options);
parameter(1:4483)=x;
resnorm1=total_error(parameter(1:4483));
k=1;
count = 1;
while(1)
opt_1; 
fprintf('expectation complete');
%maximaization step
[x] = fminunc(@costfunction,parameter(1:4483),options);
parameter(1:4483)=x;
resnorm2=total_error(parameter(1:4483));
fprintf('resnorm1-resnorm2 - %f, resnorm2 - %f, k - %f',resnorm1-resnorm2,0.000001*resnorm2,k);
if((resnorm1-resnorm2)< .000001*resnorm2 & k~=1)  %% to decrease training time 
    break;
end

但现在，当我必须在有大量参数的问题上使用它时，我得到以下日志。

                                                     First-order 
 Iteration  Func-count       f(x)        Step-size       optimality
     0           1      5.31444e+10                      4.75e+14

Optimization stopped because the objective function cannot be decreased in the 
current search direction. Either the predicted change in the objective function,
or the line search interval is less than eps.
                                                   First-order 
 Iteration  Func-count       f(x)        Step-size       optimality
     0           1      5.31444e+10                      4.75e+14

Optimization stopped because the objective function cannot be decreased in the 
current search direction. Either the predicted change in the objective function,
or the line search interval is less than eps.

resnorm1-resnorm2 - 0.000000, resnorm2 - 53144.356560, k - 1.000000 
                                                    First-order 
 Iteration  Func-count       f(x)        Step-size       optimality
     0           1      5.31444e+10                      4.75e+14

Optimization stopped because the objective function cannot be decreased in the 
current search direction. Either the predicted change in the objective function,
or the line search interval is less than eps.

    resnorm1-resnorm2 - 0.000000, resnorm2 - 53144.356560, k - 2.000000
>>

然后该过程以非常糟糕的结果完成。至于，可以看出fminunc无法正确优化。有人能帮我一下吗？

Answer 1

看起来您需要降低训练速度系数，或者将参数矢量标准化。

Answer 2

我将参数初始化从零更改为rand，并且与规范化一起，我让它工作。

混合专家 - fminunc优化过早停止

2 个答案: