我的网络仅达到80%左右,但报告的最佳分数约为85%。我使用相同的输入数据和相同的初始化。我不知道什么是错的,所以我尝试检查我的渐变并实施渐变检查的建议:http://ufldl.stanford.edu/tutorial/supervised/DebuggingGradientChecking/
但我不确定,如果我的实施是正确的:
public void gradientchecking(double[] theta){
System.out.println("Gradient Checking started");
//costfunction returns cost and gradients
IPair<Double, double[]> org = costfunction(theta);
double[] theta_pos = new double[theta.length];
double[] theta_neg = new double[theta.length];
for (int i = 0; i < theta.length; i++) {
theta_pos[i]= theta[i];
theta_neg[i]=theta[i];
}
double mu = 1e-5;
for (int k = 0; k < 20; k++) {
theta_pos[k] = theta_pos[k] + mu;
theta_neg[k] = theta_neg[k] - mu;
IPair<Double, double[]> pos = costfunction(theta_pos);
IPair<Double, double[]> neg = costfunction(theta_neg);
System.out.println("Org: "+org.getSecond()[k] +" check:"+ ((pos.getSecond()[k]-neg.getSecond()[k])/(2*mu)));
//System.out.println("Org: "+org.getSecond()[k] +"check:"+ ((pos.getSecond()[k]-neg.getSecond()[k])/(2*mu)));
theta_pos[k] = theta_pos[k] - mu;
theta_neg[k] = theta_neg[k] + mu;
}
}
}
在刚刚初始化的theta之后我得到了以下结果:
Gradient Checking started
Cost: 1.1287071297725055 | Wrong: 124 | start: Thu Jul 30 22:57:08 CEST 2015 |end: Thu Jul 30 22:57:18 CEST 2015
Cost: 1.128707130295382 | Wrong: 124 | start: Thu Jul 30 22:57:18 CEST 2015 |end: Thu Jul 30 22:57:28 CEST 2015
Cost: 1.1287071292496391 | Wrong: 124 | start: Thu Jul 30 22:57:28 CEST 2015 |end: Thu Jul 30 22:57:38 CEST 2015
Org: 5.2287135944026004E-5 check:1.0184607936733826E-4
Cost: 1.1287071299252593 | Wrong: 124 | start: Thu Jul 30 22:57:38 CEST 2015 |end: Thu Jul 30 22:57:47 CEST 2015
Cost: 1.1287071296197628 | Wrong: 124 | start: Thu Jul 30 22:57:47 CEST 2015 |end: Thu Jul 30 22:57:56 CEST 2015
Org: 1.5274823511207024E-5 check:1.141254586229615E-4
Cost: 1.1287071299063134 | Wrong: 124 | start: Thu Jul 30 22:57:56 CEST 2015 |end: Thu Jul 30 22:58:05 CEST 2015
Cost: 1.1287071296387077 | Wrong: 124 | start: Thu Jul 30 22:58:05 CEST 2015 |end: Thu Jul 30 22:58:14 CEST 2015
Org: 1.3380293717695182E-5 check:1.0008639478696018E-4
Cost: 1.1287071297943114 | Wrong: 124 | start: Thu Jul 30 22:58:14 CEST 2015 |end: Thu Jul 30 22:58:23 CEST 2015
Cost: 1.1287071297507094 | Wrong: 124 | start: Thu Jul 30 22:58:23 CEST 2015 |end: Thu Jul 30 22:58:32 CEST 2015
Org: 2.1800899147740388E-6 check:9.980780136716263E-5
表示我的渐变计算有错误,或者是gradientchecking()方法。我不确定,有人可以帮助我吗?
答案 0 :(得分:1)
In Java arrays are reference types.
int[] arr = { 8,7,6,5,4,3,2,1,8};
int[] b = arr;
b [0] = -10;
for (int i:arr) {
System.out.print (' ');
System.out.print (i);
}
outputs -10 7 6 5 4 3 2 1 8
So i mean that you incorrectly creating arrays
double[] theta_pos = theta;
double[] theta_neg = theta;
they are just references to theta, and by changing their contents you change theta also, +mu-mu = 0. Use clone() methods while copying array.
double[] theta_pos = theta.clone();
double[] theta_neg = theta.clone();
But remember that clone may not work as you expecting in some cases, with simple(non-reference) types it works ideal. Look at this Does calling clone() on an array also clone its contents?