How to fix "ValueError: Found input variables with inconsistent numbers of samples: [10000, 60000]"?

Question

I have problem with training my code using Stochastic Gradient Descent and MNIST database.

    from sklearn.datasets import fetch_mldata
    from sklearn.linear_model import SGDClassifier

    mnist = fetch_mldata('MNIST original')
    X, y = mnist["data"], mnist["target"]

    some_digit = X[36000]
    some_digit_image = some_digit.reshape(28, 28)

    X_train, X_train, y_train, y_test = X[:60000], X[60000:], y[:60000], y[60000:]


    y_train_5 = (y_train == 5)
    y_test_5 = (y_test == 5)

    sgd_clf = SGDClassifier(random_state=42)
    sgd_clf.fit(X_train, y_train_5)

Error at the end of process (in my opinion the last verse of code is bad):

  ValueError: Found input variables with inconsistent numbers of samples: [10000, 60000]

Answer 1

这是您的错字，您两次分配给X_train：

X_train, X_train, y_train, y_test = X[:60000], X[60000:], y[:60000], y[60000:]

正确的答案是：

X_train, X_test, y_train, y_test = X[:60000], X[60000:], y[:60000], y[60000:]

顺便说一句。 fetch_mldata即将被弃用，最好使用：

from sklearn.datasets import fetch_openml
X, y = fetch_openml("mnist_784", version=1, return_X_y=True)

Answer 2

我建议在训练和测试数据集之间使用分层划分，因为某些类在训练中可能会歪曲表示形式。

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)