Question

我正在尝试执行kfold交叉验证。但由于某种原因，它卡在这里，不会从这里accuracies = cross_val_score(estimator = classifier, X = X_train, y = y_train, cv = 10, n_jobs = -1)终止我不明白是什么问题。以及如何解决。

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd


dataset = pd.read_csv('Churn_Modelling.csv')
X = dataset.iloc[:, 3:13].values
y = dataset.iloc[:, 13].values

# Encoding categorical data
from sklearn.preprocessing import LabelEncoder, OneHotEncoder
labelencoder_X_1 = LabelEncoder()
X[:, 1] = labelencoder_X_1.fit_transform(X[:, 1])
labelencoder_X_2 = LabelEncoder()
X[:, 2] = labelencoder_X_2.fit_transform(X[:, 2])
onehotencoder = OneHotEncoder(categorical_features = [1])
X = onehotencoder.fit_transform(X).toarray()
X = X[:,1:]

# Splitting the dataset into the Training set and Test set
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0)

# Feature Scaling
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)


import keras
from keras.models import Sequential #Required to initialize the ANN
from keras.layers import Dense #Build layers of ANN
from keras.layers import Dropout


# Evaluating the ANN
import keras
from keras.wrappers.scikit_learn import KerasClassifier
from sklearn.model_selection import cross_val_score

from keras.models import Sequential #Required to initialize the ANN
from keras.layers import Dense #Build layers of ANN

def build_classifier(): # Builds the architecture, or the classifier
    classifier = Sequential()
    classifier.add(Dense(activation = 'relu', input_dim = 11, units = 6, kernel_initializer = 'uniform'))# add layers
    classifier.add(Dense(activation = 'relu', units = 6, kernel_initializer = 'uniform'))# add layers
    classifier.add(Dense(activation = 'sigmoid', units = 1, kernel_initializer = 'uniform'))
    classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
    return classifier
classifier = KerasClassifier(build_fn = build_classifier, batch_size = 10, nb_epoch = 100)
accuracies  = cross_val_score(estimator  = classifier, X = X_train, y = y_train, cv = 10, n_jobs = -1)
mean = accuracies.mean()
variance = accuracies.std()

编辑
我在Windows 10上将Anaconda与python 3.6配合使用。
数据集：Drive Link for dataset
当我将n_jobs = 1设置为完美时，但当n_jobs = -1时则无效。

Answer 1

由于设置了n_jobs = -1，所以将按照here中提到的文档使用所有CPU。但是，您必须了解利用所有CPU并不一定会减少执行时间，因为：

创建资源并将资源分配给新线程会带来开销。
此外，可能还存在其他瓶颈，例如数据量太大，无法同时向所有线程广播，通过RAM抢占线程（或其他资源等），如何将数据压入每个线程等。。
Python中的多线程也有许多缺点，请参见here和here。

您可以检查出与GridSearchCV和并行化here in this answer类似的问题。

此外，如@ncfith所提到的，目前没有解决此问题的方法。

参考

kfold交叉验证不会终止，停留在cross_val_score

1 个答案: