Question

在n_jobs = -1中使用sklearn.model_selection.cross_val_score作为参数时出现错误。我是深度学习和ANN的初学者，并且根据this课程的k折交叉验证中的讲师，使用n_jobs = -1使用所有CPU处理器以减少时间，但是就我而言，这是一个错误。

错误-

BrokenProcessPool：任务无法反序列化。请确保该函数的参数都是可拾取的。

可以找到here的完整堆栈跟踪。

import keras
from keras.wrappers.scikit_learn import KerasClassifier
from sklearn.model_selection import cross_val_score
def build_classifier():
    classifier = Sequential()
    classifier.add(Dense(units = 6, kernel_initializer = 'uniform', activation = 'relu', input_dim = 11))
    classifier.add(Dense(units = 6, kernel_initializer = 'uniform', activation = 'relu'))
    classifier.add(Dense(units = 1, kernel_initializer = 'uniform', activation = 'sigmoid'))
    classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
    return classifier
classifier = KerasClassifier(build_fn = build_classifier, batch_size = 10, nb_epoch = 100)
accuracies = cross_val_score(estimator = classifier, X = X_train, y = Y_train, cv = 10, n_jobs = -1)

Answer 1

n_jobs = -1不起作用，因为您的GPU关闭或我说GPU没有激活，您可以使用cuda命令激活它或使用这些链接激活tensorflow-gpu

Install Tensorflow (GPU version) for Windows and Anaconda

How to install Tensorflow-GPU on Windows 10

或者，您可以参考此内容，以了解n_jobs

Glossary of Common Terms and API Elements-term-n-jobs

Answer 2

尝试在外部文件中创建build_classifier函数并将其导入。例如：

在文件classifier_builder.py中：

import keras

def build_classifier():
  classifier = Sequential()
  classifier.add(Dense(units = 6, kernel_initializer = 'uniform', activation = 'relu', input_dim = 11))
  classifier.add(Dense(units = 6, kernel_initializer = 'uniform', activation = 'relu'))
  classifier.add(Dense(units = 1, kernel_initializer = 'uniform', activation = 'sigmoid'))
  classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
return classifier

然后在笔记本中：

import classifier_builder

classifier = KerasClassifier(build_fn = build_classifier, batch_size = 10, nb_epoch = 100)
accuracies = cross_val_score(estimator = classifier, X = X_train, y = Y_train, cv = 10, n_jobs = -1)

这为我解决了这个问题。显然，内联函数是不可腌制的。

Answer 3

我使用 n_jobs=1 跳过了错误

在cross_val_score中使用n_jobs参数时出现BrokenProcessPool

3 个答案: