Question

我的网络仅达到80％左右，但报告的最佳分数约为85％。我使用相同的输入数据和相同的初始化。我不知道什么是错的，所以我尝试检查我的渐变并实施渐变检查的建议：http://ufldl.stanford.edu/tutorial/supervised/DebuggingGradientChecking/

但我不确定，如果我的实施是正确的：

        public void gradientchecking(double[] theta){
        System.out.println("Gradient Checking started");
        //costfunction returns cost and gradients
        IPair<Double, double[]> org = costfunction(theta);
        double[] theta_pos = new double[theta.length];
        double[] theta_neg = new double[theta.length];
        for (int i = 0; i < theta.length; i++) {
            theta_pos[i]= theta[i];
            theta_neg[i]=theta[i];
        }

        double mu = 1e-5;
        for (int k = 0; k < 20; k++) {
            theta_pos[k] = theta_pos[k] + mu;
            theta_neg[k] = theta_neg[k] - mu;
            IPair<Double, double[]> pos = costfunction(theta_pos);
            IPair<Double, double[]> neg = costfunction(theta_neg);
            System.out.println("Org: "+org.getSecond()[k] +" check:"+ ((pos.getSecond()[k]-neg.getSecond()[k])/(2*mu)));
            //System.out.println("Org: "+org.getSecond()[k] +"check:"+ ((pos.getSecond()[k]-neg.getSecond()[k])/(2*mu)));
            theta_pos[k] = theta_pos[k] - mu;
            theta_neg[k] = theta_neg[k] + mu;
        }
    }
}

在刚刚初始化的theta之后我得到了以下结果：

Gradient Checking started
Cost: 1.1287071297725055 | Wrong: 124 | start: Thu Jul 30 22:57:08 CEST 2015 |end: Thu Jul 30 22:57:18 CEST 2015
Cost: 1.128707130295382 | Wrong: 124 | start: Thu Jul 30 22:57:18 CEST 2015 |end: Thu Jul 30 22:57:28 CEST 2015
Cost: 1.1287071292496391 | Wrong: 124 | start: Thu Jul 30 22:57:28 CEST 2015 |end: Thu Jul 30 22:57:38 CEST 2015
Org: 5.2287135944026004E-5 check:1.0184607936733826E-4
Cost: 1.1287071299252593 | Wrong: 124 | start: Thu Jul 30 22:57:38 CEST 2015 |end: Thu Jul 30 22:57:47 CEST 2015
Cost: 1.1287071296197628 | Wrong: 124 | start: Thu Jul 30 22:57:47 CEST 2015 |end: Thu Jul 30 22:57:56 CEST 2015
Org: 1.5274823511207024E-5 check:1.141254586229615E-4
Cost: 1.1287071299063134 | Wrong: 124 | start: Thu Jul 30 22:57:56 CEST 2015 |end: Thu Jul 30 22:58:05 CEST 2015
Cost: 1.1287071296387077 | Wrong: 124 | start: Thu Jul 30 22:58:05 CEST 2015 |end: Thu Jul 30 22:58:14 CEST 2015
Org: 1.3380293717695182E-5 check:1.0008639478696018E-4
Cost: 1.1287071297943114 | Wrong: 124 | start: Thu Jul 30 22:58:14 CEST 2015 |end: Thu Jul 30 22:58:23 CEST 2015
Cost: 1.1287071297507094 | Wrong: 124 | start: Thu Jul 30 22:58:23 CEST 2015 |end: Thu Jul 30 22:58:32 CEST 2015
Org: 2.1800899147740388E-6 check:9.980780136716263E-5

表示我的渐变计算有错误，或者是gradientchecking（）方法。我不确定，有人可以帮助我吗？

Answer 1

In Java arrays are reference types.

int[] arr = { 8,7,6,5,4,3,2,1,8};
int[] b = arr;
b [0] = -10;
for (int i:arr) {
    System.out.print (' ');
    System.out.print (i);
}

outputs -10 7 6 5 4 3 2 1 8

So i mean that you incorrectly creating arrays

double[] theta_pos = theta;
double[] theta_neg = theta;

they are just references to theta, and by changing their contents you change theta also, +mu-mu = 0. Use clone() methods while copying array.

double[] theta_pos = theta.clone();
double[] theta_neg = theta.clone();

But remember that clone may not work as you expecting in some cases, with simple(non-reference) types it works ideal. Look at this Does calling clone() on an array also clone its contents?

我的渐变检查方法是否正确并且我的渐变计算错误，反之亦然？

1 个答案: