Question

我正在使用带有TensorFlow后端的Keras来训练LSTM网络以获取一些按时间顺序排列的数据集。当我以Numpy数组格式表示训练数据（以及验证数据）时，性能似乎还不错：

train_x.shape: (128346, 10, 34)
val_x.shape: (7941, 10, 34)
test_x.shape: (24181, 10, 34)
train_y.shape: (128346, 2)
val_y.shape: (7941, 2)
test_y.shape: (24181, 2)

P.s。，时间步长为10，要素数量为34；标签是一键编码的。

model = tf.keras.Sequential()
model.add(layers.LSTM(_HIDDEN_SIZE, return_sequences=True, 
          input_shape=(_TIME_STEPS, _FEATURE_DIMENTIONS)))
model.add(layers.Dropout(0.4))
model.add(layers.LSTM(_HIDDEN_SIZE, return_sequences=True))
model.add(layers.Dropout(0.3))
model.add(layers.TimeDistributed(layers.Dense(_NUM_CLASSES)))
model.add(layers.Flatten())
model.add(layers.Dense(_NUM_CLASSES, activation='softmax'))

opt = tf.keras.optimizers.Adam(lr = _LR)
model.compile(optimizer = opt, loss = 'categorical_crossentropy', 
              metrics = ['accuracy'])

model.fit(train_x, 
          train_y, 
          epochs=_EPOCH, 
          batch_size = _BATCH_SIZE, 
          verbose = 1, 
          validation_data = (val_x, val_y)
         )

培训结果为：

Train on 128346 samples, validate on 7941 samples
Epoch 1/10
128346/128346 [==============================] - 50s 390us/step - loss: 0.5883 - acc: 0.6975 - val_loss: 0.5242 - val_acc: 0.7416
Epoch 2/10
128346/128346 [==============================] - 49s 383us/step - loss: 0.4804 - acc: 0.7687 - val_loss: 0.4265 - val_acc: 0.8014
Epoch 3/10
128346/128346 [==============================] - 49s 383us/step - loss: 0.4232 - acc: 0.8076 - val_loss: 0.4095 - val_acc: 0.8096
Epoch 4/10
128346/128346 [==============================] - 49s 383us/step - loss: 0.3894 - acc: 0.8276 - val_loss: 0.3529 - val_acc: 0.8469
Epoch 5/10
128346/128346 [==============================] - 49s 382us/step - loss: 0.3610 - acc: 0.8430 - val_loss: 0.3283 - val_acc: 0.8593
Epoch 6/10
128346/128346 [==============================] - 49s 382us/step - loss: 0.3402 - acc: 0.8525 - val_loss: 0.3334 - val_acc: 0.8558
Epoch 7/10
128346/128346 [==============================] - 49s 383us/step - loss: 0.3233 - acc: 0.8604 - val_loss: 0.2944 - val_acc: 0.8741
Epoch 8/10
128346/128346 [==============================] - 49s 383us/step - loss: 0.3087 - acc: 0.8663 - val_loss: 0.2786 - val_acc: 0.8805
Epoch 9/10
128346/128346 [==============================] - 49s 383us/step - loss: 0.2969 - acc: 0.8709 - val_loss: 0.2785 - val_acc: 0.8777
Epoch 10/10
128346/128346 [==============================] - 49s 383us/step - loss: 0.2867 - acc: 0.8757 - val_loss: 0.2590 - val_acc: 0.8877

该日志似乎很正常，但是当我尝试使用TensorFlow Dataset API表示我的数据集时，训练过程执行得很奇怪（似乎模型变成了过拟合/欠拟合？）：

def tfdata_generator(features, labels, is_training = False, batch_size = _BATCH_SIZE, epoch = _EPOCH):
    dataset = tf.data.Dataset.from_tensor_slices((features, tf.cast(labels, dtype = tf.uint8)))
    if is_training:
        dataset = dataset.shuffle(10000)  # depends on sample size
    dataset = dataset.batch(batch_size, drop_remainder = True).repeat(epoch).prefetch(batch_size)
    return dataset

training_set = tfdata_generator(train_x, train_y, is_training=True)
validation_set = tfdata_generator(val_x, val_y, is_training=False)
testing_set  = tfdata_generator(test_x, test_y, is_training=False)

训练相同的模型和超参数：

model.fit(
          training_set.make_one_shot_iterator(), 
          epochs = _EPOCH,
          steps_per_epoch = len(train_x) // _BATCH_SIZE,
          verbose = 1, 
          validation_data = validation_set.make_one_shot_iterator(),
          validation_steps =  len(val_x) // _BATCH_SIZE
         )

日志似乎与上一个有很大不同：

Epoch 1/10
2005/2005 [==============================] - 54s 27ms/step - loss: 0.1451 - acc: 0.9419 - val_loss: 3.2980 - val_acc: 0.4975
Epoch 2/10
2005/2005 [==============================] - 49s 24ms/step - loss: 0.1675 - acc: 0.9371 - val_loss: 3.0838 - val_acc: 0.4975
Epoch 3/10
2005/2005 [==============================] - 49s 24ms/step - loss: 0.1821 - acc: 0.9316 - val_loss: 3.1212 - val_acc: 0.4975
Epoch 4/10
2005/2005 [==============================] - 49s 24ms/step - loss: 0.1902 - acc: 0.9287 - val_loss: 3.0032 - val_acc: 0.4975
Epoch 5/10
2005/2005 [==============================] - 49s 24ms/step - loss: 0.1905 - acc: 0.9283 - val_loss: 2.9671 - val_acc: 0.4975
Epoch 6/10
2005/2005 [==============================] - 49s 24ms/step - loss: 0.1867 - acc: 0.9299 - val_loss: 2.8734 - val_acc: 0.4975
Epoch 7/10
2005/2005 [==============================] - 49s 24ms/step - loss: 0.1802 - acc: 0.9316 - val_loss: 2.8651 - val_acc: 0.4975
Epoch 8/10
2005/2005 [==============================] - 49s 24ms/step - loss: 0.1740 - acc: 0.9350 - val_loss: 2.8793 - val_acc: 0.4975
Epoch 9/10
2005/2005 [==============================] - 49s 24ms/step - loss: 0.1660 - acc: 0.9388 - val_loss: 2.7894 - val_acc: 0.4975
Epoch 10/10
2005/2005 [==============================] - 49s 24ms/step - loss: 0.1613 - acc: 0.9405 - val_loss: 2.7997 - val_acc: 0.4975

使用TensorFlow Dataset API表示数据时，无法减少验证损失，并且val_acc始终保持相同的值。

我的问题是：

基于相同的模型和参数，为什么当我仅采用tf.data.Dataset API时，model.fit（）提供如此不同的训练结果？

这两种机制有什么区别？

   model.fit(train_x, 
      train_y, 
      epochs=_EPOCH, 
      batch_size = _BATCH_SIZE, 
      verbose = 1, 
      validation_data = (val_x, val_y)
     )

vs

    model.fit(
      training_set.make_one_shot_iterator(), 
      epochs = _EPOCH,
      steps_per_epoch = len(train_x) // _BATCH_SIZE,
      verbose = 1, 
      validation_data = validation_set.make_one_shot_iterator(),
      validation_steps =  len(val_x) // _BATCH_SIZE
     )

如果必须使用tf.data.Dataset API，如何解决这个奇怪的问题？

针对不同训练数据表示（Numpy数组和TensorFlow数据集API）的不同收敛

0 个答案: