创建SparkSession时getOrCreate上的空指针异常

时间:2017-11-06 20:57:03

标签: apache scala apache-spark-sql

我正在尝试在REPL上创建一个spark会话(在Linux x86_64上)并且我得到以下错误。

你能告诉我吗?

>echo $PATH
/opt/installations/spark-2.2.0-bin-hadoop2.7/bin:/opt/local/installations/scala-2.11.8/bin:/opt/env/java/latest/bin:/opt/env/java/latest/jre/bin:/opt/env/oracle/latest/bin:/opt/env/oracle/latest/network/admin:/opt/jre/1.6.0_81l64/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/X11R6/bin:/usr/local/sbin:.

能够运行直到我使用getOrCreate函数

scala> SparkSession
res0: org.apache.spark.sql.SparkSession.type = org.apache.spark.sql.SparkSession$@47af7f3d

scala> SparkSession.builder
res1: org.apache.spark.sql.SparkSession.Builder = org.apache.spark.sql.SparkSession$Builder@6aeb35e6

scala> SparkSession.builder.appName("abcd")
    res2: org.apache.spark.sql.SparkSession.Builder = org.apache.spark.sql.SparkSession$Builder@40a4337a

scala> SparkSession.builder.appName("abcd").master("local")
res3: org.apache.spark.sql.SparkSession.Builder = org.apache.spark.sql.SparkSession$Builder@7d20d0b

scala> SparkSession.builder.appName("abcd").master("local").getOrCreate()
java.lang.NullPointerException
  at org.apache.spark.internal.Logging$class.initializeLogging(Logging.scala:121)
  at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:102)
  at org.apache.spark.SparkContext$.initializeLogIfNecessary(SparkContext.scala:2431)
  at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
  at org.apache.spark.SparkContext$.log(SparkContext.scala:2431)
  at org.apache.spark.internal.Logging$class.logWarning(Logging.scala:66)
  at org.apache.spark.SparkContext$.logWarning(SparkContext.scala:2431)
  at org.apache.spark.SparkContext$$anonfun$assertNoOtherContextIsRunning$4.apply(SparkContext.scala:2489)
  at org.apache.spark.SparkContext$$anonfun$assertNoOtherContextIsRunning$4.apply(SparkContext.scala:2480)
  at scala.Option.foreach(Option.scala:257)
  at org.apache.spark.SparkContext$.assertNoOtherContextIsRunning(SparkContext.scala:2480)
  at org.apache.spark.SparkContext$.markPartiallyConstructed(SparkContext.scala:2557)
  at org.apache.spark.SparkContext.<init>(SparkContext.scala:85)
  at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2509)
  at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:909)
  at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:901)
  at scala.Option.getOrElse(Option.scala:121)
  at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:901)
  ... 32 elided

0 个答案:

没有答案