Scala / Spark中的Graphframe错误

时间:2017-10-16 08:00:55

标签: scala apache-spark databricks graphframes

我在Scala 2.11中将这些代码行写入Databricks:

import org.graphframes._

val user_ridotto = sqlContext.sql("SELECT * FROM userRidotto")

var users_1 = user_ridotto.select("user_id", "name", "city", "num_fr",  
"fans", "review_count", "importance").withColumnRenamed("user_id", "id")
val users = users_1.withColumn("ridotto", lit("ridotto"))
var edges_1 = user_ridotto.select($"user_id" as "src", explode($"friends") 
as 
"dst", lit(1))
val graph_1 = GraphFrame(users, edges_1)
println("users: " + users.count + ", archi_1: " + edges_1.count)

val paths = graph_1.find("(a)-[e]->(b)")
.filter("a.ridotto='ridotto'")
.filter("b.ridotto='ridotto'")
val edges = paths.select("e.src", "e.dst")
val graph = GraphFrame(graph_1.vertices, edges)
println("users: " + users.count + ", archi: " + edges.count)

但它返回了我这些错误:

notebook:1: error: object graphframes is not a member of package org
import org.graphframes._
           ^
notebook:8: error: not found: value GraphFrame
val graph_1 = GraphFrame(users, edges_1)
          ^
notebook:15: error: not found: value GraphFrame
val graph = GraphFrame(graph_1.vertices, edges)

提前致谢 诚挚 安东尼奥

1 个答案:

答案 0 :(得分:0)

库没有天真地编码,所以你需要在库下导入jar文件。你可以在工作区下导入文件(应该有一个允许你专门导入jar文件的消息框),然后附加它。附加后,您需要重新启动群集

您可以在graphframes

获取jar文件