在Centos群集上使用带有火花的elephas和keras - theano后端

时间:2016-11-24 08:27:20

标签: apache-spark centos pyspark deep-learning keras

我可以在AWS独立版上使用elephas,theano后端运行代码进行训练和评分模型,但我无法在安装了spark 1.6的Centos群集上进行此操作。我收到错误'模型需要在被使用之前在此行编译':spark_model.train(rdd,nb_epoch = nb_epoch,batch_size = batch_size,verbose = 2,validation_split = 0.1)

elepha文档中提供的示例代码:

conf = SparkConf().setAppName('Mnist_Spark_MLP').setMaster('local[8]')
sc = SparkContext(conf=conf)
model = Sequential()
model.add(Dense(128, input_dim=784))
model.add(Activation('relu'))
model.add(Dropout(0.2))
model.add(Dense(128))
model.add(Activation('relu'))
model.add(Dropout(0.2))
model.add(Dense(10))
model.add(Activation('softmax'))
sgd = SGD(lr=0.1)
rdd = to_simple_rdd(sc, x_train, y_train)
adagrad = elephas_optimizers.Adagrad()
spark_model = SparkModel(sc,
                         model,
                         optimizer=adagrad,
                         frequency='epoch',
                         mode='asynchronous',
                         num_workers=2,master_optimizer=sgd)
spark_model.train(rdd, nb_epoch=nb_epoch, batch_size=batch_size, verbose=2, validation_split=0.1)

解决此问题的方法是什么?

0 个答案:

没有答案