在HDFS

时间:2016-09-30 12:47:40

标签: scala apache-spark hdfs k-means bigdata

我想在Hdfs上保存KMeans模型。为此,我使用方法save并在运行时创建otuput目录(参见代码)。我得到错误异常元数据已经存在。我该如何解决这个问题?

  val lastUrbanKMeansModel = KMeansModel.load(spark, defaultPath + "UrbanRoad/201692918")
  val newUrbanKMeansObject = new KMeans()
    .setK(7)
    .setMaxIterations(20)
    .setInitialModel(lastUrbanKMeansModel)
  val vectorUrbanRoad = typeStreet.filter(k => k._2 == 1).map(_._1)

  if (!vectorUrbanRoad.isEmpty()) {

    val newUrbanModel = newUrbanKMeansObject.run(vectorUrbanRoad)
    newUrbanModel.save(spark, defaultPath + "UrbanRoad/" + 
      Calendar.getInstance().get(Calendar.YEAR).toString
      + (Calendar.getInstance().get(Calendar.MONTH) + 1).toString + 
      Calendar.getInstance().get(Calendar.DAY_OF_MONTH).toString + 
      Calendar.getInstance().get(Calendar.HOUR_OF_DAY).toString)
  } 

0 个答案:

没有答案