在Spark中使用KerasClassifier进行交叉验证时出现PicklingError

时间:2018-11-23 22:54:54

标签: keras pyspark

尽管我能够在本地计算机上完美运行以下代码:

`from keras.wrappers.scikit_learn import KerasClassifier
 from sklearn.model_selection import cross_val_score
 from keras.models import Sequential
 from keras.layers import Dense
 def build_classifier():
     classifier = Sequential()
     classifier.add(Dense(units = 6, kernel_initializer = 'uniform', activation = 'relu', input_dim = 11))
     classifier.add(Dense(units = 6, kernel_initializer = 'uniform', activation = 'relu'))
     classifier.add(Dense(units = 1, kernel_initializer = 'uniform', activation = 'sigmoid'))
     classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
     return classifier
classifier = KerasClassifier(build_fn = build_classifier, batch_size = 10, epochs = 100)
accuracies = cross_val_score(estimator = classifier, X = X_train, y = y_train, cv = 10, n_jobs = -1)`

我得到PicklingErrorCan't pickle <function build_classifier at 0x7f98f3cbabf8>: attribute lookup build_classifier on __main__ failed

基于我对PicklingError进行并行化时对Python的理解,看来我在定义函数的位置上存在一些问题,但我无法做到跑。我该如何解决这个问题?

0 个答案:

没有答案