在火花中,将partialFunction替换为另一个函数出错

时间:2018-08-13 11:39:27

标签: scala apache-spark partialfunction

这是我的代码,是pageRank算法的体现:

template<typename KeyT, typename ValueT,
         typename MapType = llvm::DenseMap<KeyT, unsigned>,
         typename VectorType = std::vector<std::pair<KeyT, ValueT> > >
class MapVector {
...
}

我想使用下面的函数用flatmap替换partialFunction:

def pageRank(sc: SparkContext): Unit = {
  //Init
  val links = xxx//data type: String, List[String]
  var ranks = links.mapValues(_ => 1.0)

  //Iteration
  for (i <- 0 until iterCnt) {
  val contributions = links.join(ranks)
    //use partialFunc
    .flatMap{
    case (_, (linkList, rank)) =>
      val tuples = linkList.map(dest => (dest, rank / linkList.size))
      tuples
  }
  ranks = contributions.reduceByKey((x, y) => x + y)
    .mapValues(v => {
      (1 - alpha) + alpha * v
    })
  }
}

但是我收到“类型不匹配”错误:  找到:(String,(List [String],Double))=> List [(String,Double)]  required:(((String,(List [String],Double))))=> TraversableOnce [?]

我错过了什么吗?

0 个答案:

没有答案