培训过程

Question

当我训练一个Resnet时，改组后的验证集的准确性不等于未改组后的版本。训练集以相同的方式随机播放。

我尝试每次都使用一个样本来馈送网络，并且在vali集中获得相同的准确性。但是，当我每次使用一批样本（例如256个）时，精度是不相等的。

培训过程

count = 1 
while 1:

    #get training_data
    if (count - 1) % 5200 == 0:
        train_set,train_label = get_traindata(eng, 70)

    tset,tlabel = train_set,train_label
    tset_num = tset.shape[0]

    #split and train
    num_batches = tset_num // batch_size
    order = np.random.choice(tset_num, num_batches * batch_size,replace = False)
    train_data_subset = tset[order,:]
    train_labels_subset = tlabel[order]

    for step in range(num_batches):

        offset = step * validation_size
        feed_dict = {
                    x: train_data_subset[offset:offset+batch_size,:],
                    y: train_labels_subset[offset:offset+batch_size],
                    }
        _, tloss, tacc,gd = sess.run([train_op,loss, accuracy,grad_norm], 
                                     feed_dict=feed_dict)
        if (count-1) % 200 == 0:


            #get validationset
            if (count-1) % 1400 == 0:
                vset,vlabel = get_validata(eng, 20)
                vnum_batches = vset.shape[0] // validation_size
                order = np.random.choice(vset.shape[0], vnum_batches * validation_size,replace = False)  #shuffled
                va_data_subset = vset[order,:]
                va_labels_subset = vlabel[order]

            loss_list = []
            error_list = []
            loss_list_s = []
            error_list_s = []
            for stepv in range(vnum_batches):
               offsetv = stepv * validation_size
               feed_dict_shuffle = {
                    x: va_data_subset[offsetv:offsetv+validation_size,:],
                    y: va_labels_subset[offsetv:offsetv+validation_size],
                    }
               vloss_s, vacc_s = sess.run([loss, accuracy], feed_dict=feed_dict_shuffle)
               loss_list_s.append(vloss_s)
               error_list_s.append(vacc_s)
               feed_dict = {
                    x: vset[offsetv:offsetv+validation_size,:],
                    y: vlabel[offsetv:offsetv+validation_size],
                    }
               vloss, vacc = sess.run([loss, accuracy], feed_dict=feed_dict)
               loss_list.append(vloss)
               error_list.append(vacc)
            #the performance in the shuffled vali-set
            vloss_s = np.mean(loss_list_s)
            vacc_s = np.mean(error_list_s)
            #the performance in the unshuffled vali-set
            vloss = np.mean(loss_list)
            vacc = np.mean(error_list)

从理论上讲，变量集的顺序不会影响结果。

Answer 1

我假设您正在使用辍学或批处理规范化。无论哪种情况，您都需要在推断过程中将bool tensorflow变量输入训练参数。由于顺序似乎会影响结果，因此我将假定批量标准化：
https://www.tensorflow.org/api_docs/python/tf/layers/batch_normalization。
feed training =使用张量流占位符为假

如何修复在验证步骤中使用的代码？

培训过程

1 个答案: