我来自Hive的静态DataFrame是:
val pageCategoryDf = sql("select * from cnbo.page_category")
val userProfileDf = sql("select * from cnbo.user_profile_g")
val productProfileDf = sql("select * from cnbo.product_profile")
hiveDf
是我来自Kafka的流数据帧。我正在执行Joins,如下所示:
val joinedDf1 = pageCategoryDf.join(hiveDf, $"pageurl" === $"endpoint","right_outer")
val joinedDf2 = userProfileDf.join(joinedDf1 ,$"ipaddress" === $"ip","right_outer")
val joinedDf3 = productProfileDf.join(joinedDf2, $"producturl" === $"endpoint","right_outer")
运行完整的流查询时,出现以下错误:
java.lang.AssertionError:断言失败:没有计划 MetastoreRelation cnbo,user_profile_g
我正在使用Spark 2.1。