我无法找到一种在缩放中对TypedPipe
的元素进行排序的方法(不执行group
操作时)。这是我程序的相关部分(用省略号替换不相关的部分):
case class ReduceOutput(val slug : String, score : Int, json1 : String, json2 : String)
val pipe1 : TypedPipe[(String, ReduceFeatures)] = ...
val pipe2 : TypedPipe[(String, ReduceFeatures)] = ...
pipe1.join(pipe2).map { entry =>
val (slug : String, (features1 : ReduceFeatures, features2 : ReduceFeatures)) = entry
new ReduceOutput(
slug,
computeScore(features1, features2),
features1.json,
features2.json)
}
.write(TypedTsv[ReduceOutput](args("output")))
是否可以对score
之后但map
之前的write
上的元素进行排序?