我有以下代码从pyspark连接到蜂巢。但是我遇到了这个错误,
AnalysisException:'java.lang.RuntimeException:java.io.IOException:(null)命令字符串中的条目:null chmod 0700 C:\ Users \ deepak \ AppData \ Local \ Temp \ 006a8f0d-4fde-473a-951d- ae1b6c3f1e45_resources;' 这就是我要连接的东西。
from pyspark import SparkContext, SparkConf
from pyspark.sql import SparkSession, HiveContext,SQLContext
SparkContext.setSystemProperty("hive.metastore.uris", "thrift://x.x.x.x:9083")
SparkContext.setSystemProperty("spark.hadoop.fs.defaultFS", "hdfs://x.x.x.x:8020")
SparkContext.setSystemProperty("spark.yarn.jars", "hdfs://x.x.x.x/user/talentorigin/jars/*.jar")
SparkContext.setSystemProperty("spark.hadoop.yarn.resourcemanager.address", "x.x.x.x:8032")
sparkSession = (SparkSession
.builder
.appName('Pyspark Read from hive')
.enableHiveSupport()
.getOrCreate())
df_load=sparkSession.sql('Show databases')
df_load.show()