从Jupyter / Pyspark运行Hive SQL时发生异常

时间:2018-12-30 06:56:33

标签: hive pyspark jupyter

我已经在Mac中安装了spark。根据文档here,“没有现有Hive部署的用户仍可以启用Hive支持”。我的spark / conf目录中有hive-site.xml。我可以看到在我的工作目录中创建了derby.log和metastore_db。

我在Jupyter笔记本中的pyspark代码如下:

spark = SparkSession.builder.appName("aov").config("spark.sql.hive.metastore.jars", "maven").enableHiveSupport().getOrCreate()

sqlContext.sql(“显示表格”)

我遇到以下异常:

Caused by: org.datanucleus.exceptions.NucleusException: Attempt to invoke the "HikariCP" plugin to create a ConnectionPool gave an error : The connection pool plugin of type "HikariCP" was not found in the CLASSPATH!
at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:259)
at org.datanucleus.store.rdbms.ConnectionFactoryImpl.initialiseDataSources(ConnectionFactoryImpl.java:131)
at org.datanucleus.store.rdbms.ConnectionFactoryImpl.<init>(ConnectionFactoryImpl.java:85)
... 112 more

由以下原因引起:org.datanucleus.exceptions.NucleusUserException:在CLASSPATH中找不到类型为“ HikariCP”的连接池插件!     在org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:234)     ...还有114个

有人可以帮我吗?

0 个答案:

没有答案