Spark ALSModel方法predictForSetOfUser需要太多时间

时间:2019-04-11 19:43:41

标签: apache-spark apache-spark-mllib

val cfModelHdfs: ALSModel = ALSModel.load(outputPathHdfs)
cfModelHdfs.userFactors.cache
cfModelHdfs.itemFactors.cache

val currentUserPrediction = predictForSetOfUser(cfModelHdfs, userIndexedIdDf, modelParams.numUserPrediction)

I have aroung 13 mil user and 4 mil item and this method [predictForSetOfUser] takes around 12 hour



EMR cluster 6TB memory

如何改善此算法的运行时间

0 个答案:

没有答案