Question

我是RDD和图表的新手，我使用Pregel计算Graph中的最短路径。我想根据labels.get(vertex._1)

对最终结果进行排序

public static void shortestPaths(JavaSparkContext ctx) {
        Map<Long, String> labels = ImmutableMap.<Long, String>builder()
                .put(1l, "A")
                .put(2l, "B")
                .put(3l, "C")
                .build();
    List<Tuple2<Object,Integer>> vertices = Lists.newArrayList(
            new Tuple2<Object,Integer>(1l,0),
            new Tuple2<Object,Integer>(2l,Integer.MAX_VALUE),
            new Tuple2<Object,Integer>(3l,Integer.MAX_VALUE),

    );
    List<Edge<Integer>> edges = Lists.newArrayList(
            new Edge<Integer>(1l,2l, 4), // A --> B (4)
            new Edge<Integer>(1l,3l, 8), // A --> C (8)
            new Edge<Integer>(2l,3l, 5), // B --> C (5)

    );

    JavaRDD<Tuple2<Object,Integer>> verticesRDD = ctx.parallelize(vertices);
    JavaRDD<Edge<Integer>> edgesRDD = ctx.parallelize(edges);

    Graph<Integer,Integer> G = Graph.apply(verticesRDD.rdd(),edgesRDD.rdd(),1, StorageLevel.MEMORY_ONLY(), StorageLevel.MEMORY_ONLY(),
            scala.reflect.ClassTag$.MODULE$.apply(Integer.class),scala.reflect.ClassTag$.MODULE$.apply(Integer.class));

    GraphOps ops = new GraphOps(G, scala.reflect.ClassTag$.MODULE$.apply(Integer.class),scala.reflect.ClassTag$.MODULE$.apply(Integer.class));

    ops.pregel(Integer.MAX_VALUE,
            Integer.MAX_VALUE,
            EdgeDirection.Out(),
            new VProg(),
            new sendMsg(),
            new merge(),
            ClassTag$.MODULE$.apply(Integer.class))
        .vertices()
        .toJavaRDD()
        .foreach(v -> {
Tuple2<Object,Integer> vertex = (Tuple2<Object,Integer>)v;
System.out.println("Mini cost from "+labels.get(1l)+" to "+labels.get(vertex._1)+" is "+vertex._2);

}）;

其实我的结果是：

Min cost from A to B is 4
Min cost from A to A is 0
Min cost from A to C is 8

我想要

    Min cost from A to A is 0
    Min cost from A to B is 4
    Min cost from A to C is 8

你能帮我解决一下如何对JavaRDD进行排序。

谢谢。

Answer 1

只需在.sortBy(f -> ((Tuple2<Object, Integer>) f)._1, true, 0)之后添加.toJavaRDD()。

希望它有所帮助！

使用Pregel作为最短路径时对JavaRDD进行排序

1 个答案: