如何在给定多个顶点类型的情况下加入spark graphx

Question

我对sparkx相对较新。基本上我的图表有：

1. 2个顶点类型：人和车
2. edge描述哪个人拥有哪辆车

我想在图表中给出所有人物顶点，遍历边缘以收集每个人的汽车列表

e.g。

person1 -> [car1, car2]
person2 -> [car3]

Answer 1

您可以通过一些SQL实现此目的。

假设您有以下图表：

import org.apache.spark.graphx
import org.apache.spark.rdd.RDD

// Create an RDD for the vertices
val v: RDD[(VertexId, (String))] =
  sc.parallelize(Array((1L, ("car1")), (2L, ("car2")),
                       (3L, ("car3")), (4L, ("person1")),(5L, ("person2"))))
// Create an RDD for edges
val e: RDD[Edge[Int]] =
  sc.parallelize(Array(Edge(4L, 1L,1),    Edge(4L, 2L, 1),
                       Edge(5L, 1L,1)))


val graph = Graph(v,e)

现在将边和顶点提取到Dataframes：

val vDf = graph.vertices.toDF("vId","vName")
val eDf =graph.edges.toDF("person","car","attr")

将数据转换为所需的输出

eDf.drop("attr").join(vDf,'person === 'vId).drop("vId","person").withColumnRenamed("vName","person")
.join(vDf,'car === 'vId).drop("car","vId")
.groupBy("person")
.agg(collect_set('vName)).toDF("person","car")
.show()


+-------+------------+
| person|         car|
+-------+------------+
|person2|      [car1]|
|person1|[car2, car1]|
+-------+------------+