绘制训练和验证损失曲线？

Question

我正在使用酵母数据集，网址为：

http://archive.ics.uci.edu/ml/datasets/yeast

，我想建立一个神经网络分类器模型并绘制学习曲线。因此，我已经使用了scikit的model_selection两次；一个用于制作训练和测试集，另一个用于选择验证集。从这两个集合中，我想绘制学习曲线，我的代码如下：

import numpy as np
import pandas as pd
from sklearn import model_selection, linear_model
from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix
from sklearn.neural_network import MLPClassifier
import matplotlib.pyplot as plt

def readFile(file):
    head=["seq_n","mcg","gvh","alm","mit","erl","pox","vac","nuc","site"]
    f=pd.read_csv(file,delimiter=r"\s+")
    f.columns=head
    return f

def NeuralClass(X,y):
    X_train,X_test,y_train,y_test=model_selection.train_test_split(X,y,test_size=0.2)
    X_tr,X_val,y_tr,y_val=model_selection.train_test_split(X_train,y_train,test_size=0.2)
    mlp=MLPClassifier(activation="relu",max_iter=3000)
    mlp.fit(X_train,y_train)
    print (mlp.score(X_train,y_train))
    plt.plot(mlp.loss_curve_)
    mlp.fit(X_val,y_val)
    plt.plot(mlp.loss_curve_)

def main():
    f=readFile("yeast.data")
    list=["seq_n","site"]
    X=f.drop(list,1)
    y=f["site"]
    NeuralClass(X,y)

if __name__=="__main__":
    main()

我获得了如下图，我不知道它是否正确：

问题是这是否是绘制验证曲线的正确方法，或者我遵循的方法是否正确。

谢谢

Answer 1

没有测试，但是应该是这样的：

def NeuralClass(X,y):
    X_train,X_test,y_train,y_test = model_selection.train_test_split(
        X,y,test_size=0.2)
    mlp=MLPClassifier(
        activation="relu",
        max_iter=3000, 
        validation_fraction=0.2, 
        early_stopping=True)
    mlp.fit(X_train,y_train)
    print (mlp.score(X_train,y_train))
    plt.plot(mlp.loss_curve_)
    plt.plot(mlp.validation_scores_)