我有一个相当大的数据集，分为训练/验证类别。网络的目标是预测特定点是“热点”，基本上是对还是错。我大约有40个功能。但是，在运行我的基本神经网络时，我发现训练数据的成功率很容易获得正确（在某些参数下约为98％），但验证率却很低（约62％）。因为我只猜到50％，所以我对成功率不满意。
我建立了一个基本的tensorflow程序，因为我通常是新手。这是一个链接到DNNClassifier的ProximalAdagradOptimizer，具有我尝试过的不同形状的网络。
下面是培训本身的代码，不包括功能处理/等。其中很多是从Google的速成班获得的。
def train_nn_classification_model(
my_optimizer,
steps,
batch_size,
hidden_units,
training_examples,
training_targets,
validation_examples,
validation_targets):
"""Trains a neural network classification model.
In addition to training, this function also prints training progress information,
as well as a plot of the training and validation loss over time.
Args:
my_optimizer: An instance of `tf.train.Optimizer`, the optimizer to use.
steps: A non-zero `int`, the total number of training steps. A training step
consists of a forward and backward pass using a single batch.
batch_size: A non-zero `int`, the batch size.
hidden_units: A `list` of int values, specifying the number of neurons in each layer.
training_examples: A dataframe containing the features to predict the hotspot value of each instance.
training_targets: A dataframe containing one column, and the hotspot values to target.
validation_examples: A `DataFrame` containing one or more columns to use as input features for validation.
validation_targets: A `DataFrame` containing exactly one column to use as target for validation.
Returns:
A tuple `(estimator, training_losses, validation_losses)`:
estimator: the trained `DNNClassifier` object.
training_losses: a `list` containing the training loss values taken during training.
validation_losses: a `list` containing the validation loss values taken during training.
"""
periods = 10
steps_per_period = steps / periods
# Create a DNNClassifier object.
#my_optimizer = tf.contrib.estimator.clip_gradients_by_norm(my_optimizer, 5.0)
print("creating DNNClassifier object...")
dnn_classifier = tf.estimator.DNNClassifier(
feature_columns=construct_feature_columns(training_examples),
hidden_units = hidden_units,
optimizer = my_optimizer
)
# Create input functions.
print("creating input functions...")
training_input_fn = lambda: input_fn(training_examples,
training_targets["hotspot"],
batch_size=batch_size)
print("TRAINING TARGETS")
print(training_targets)
predict_training_input_fn = lambda: input_fn(training_examples,
training_targets["hotspot"],
num_epochs=1,
shuffle=False)
predict_validation_input_fn = lambda: input_fn(validation_examples,
validation_targets["hotspot"],
num_epochs=1,
shuffle=False)
# Train the model, but do so inside a loop so that we can periodically assess
# loss metrics.
print("Training model...")
print("Success Rate (on training data):")
training_success = []
validation_success = []
training_correct = 0
training_total = 0
validation_correct = 0
validation_total = 0
for period in range (0, periods):
# Train the model, starting from the prior state.
dnn_classifier.train(
input_fn=training_input_fn,
steps=steps_per_period
)
# Take a break and compute predictions.
training_predictions = dnn_classifier.predict(input_fn=predict_training_input_fn)
training_predictions = [int(item['classes']) for item in list(training_predictions)]
validation_predictions = dnn_classifier.predict(input_fn=predict_validation_input_fn)
validation_predictions = [int(item['classes']) for item in list(validation_predictions)]
# Compute training and validation success rate
for i in range(0, len(training_predictions)):
if training_predictions[i] == training_targets['hotspot'][i]:
training_correct += 1
training_total += 1
training_success_rate = training_correct / training_total
for i in range(0, len(validation_predictions)):
if validation_predictions[i] == validation_targets['hotspot'][i]:
validation_correct += 1
validation_total += 1
validation_success_rate = validation_correct / validation_total
# Occasionally print the current loss.
print(" period %02d : %0.2f" % (period, training_success_rate))
# Add the loss metrics from this period to our list.
training_success.append(training_success_rate)
validation_success.append(validation_success_rate)
print("Model training finished.")
# Output a graph of loss metrics over periods.
plt.ylabel("RMSE")
plt.xlabel("Periods")
plt.title("Root Mean Squared Error vs. Periods")
plt.tight_layout()
plt.plot(training_rmse, label="training")
plt.plot(validation_rmse, label="validation")
plt.legend()
print("Final success rate(on training data): %0.2f" % training_success_rate)
print("Final success rate (on validation data): %0.2f" % validation_success_rate)
print("Model training finished")
return dnn_classifier, training_success, validation_success
user_learning_rate = 0.007
_ = train_nn_classification_model(
my_optimizer=tf.train.ProximalAdagradOptimizer(learning_rate=user_learning_rate, l2_regularization_rate=0.03),
steps=5000,
batch_size=70,
hidden_units=[30, 20, 10, 5],
training_examples=training_examples,
training_targets=training_targets,
validation_examples=validation_examples,
validation_targets=validation_targets)
我强迫l2正则化过高，并得到了这个结果。
period 00 : 0.84
period 01 : 0.87
period 02 : 0.88
period 03 : 0.89
period 04 : 0.90
period 05 : 0.91
period 06 : 0.91
period 07 : 0.92
period 08 : 0.92
period 09 : 0.93
Final success rate(on training data): 0.93
Final success rate (on validation data): 0.61
Model training finished
该模型似乎训练得很好，但是验证率却很低。我要寻找的是一些基本技术，以尝试查看是否可以提高模型的成功率。
答案 0 :(得分：0)
这可能不是 答案，但是您的问题可能是您的功能不能很好地说明“热点”。
您可以执行以下操作进行健全性检查：
希望这会有所帮助。