Question

我安装了DC / OS并使用 dcos软件包安装--package-version = 1.0.1-1.6.1-2 spark

安装spark

Spark已成功安装，我可以在主节点上打开火花驱动程序页面。

现在，我尝试设置zeppelin并将其连接到mesos群集。我在zeppelin-env.sh中使用了下一个env变量：

export SPARK_HOME=/opt/spark-1.6.1
export MESOS_NATIVE_JAVA_LIBRARY=/usr/lib/libmesos.so 
export SPARK_EXECUTOR_URI=http://downloads.mesosphere.io.s3.amazonaws.com/spark/assets/spark-1.6.1.tgz
export ZEPPELIN_JAVA_OPTS="-Dspark.executor.uri=$SPARK_EXECUTOR_URI"

并通过zeppelin UI将spark master url设置为 mesos：// myhostname：5050

你能帮我理解出了什么问题，我的日志中有以下错误信息：

I0810 14:10:36.157269   201 sched.cpp:222] Version: 0.28.1
I0810 14:10:36.160846   192 sched.cpp:326] New master detected at master@192.168.1.22:5050
I0810 14:10:36.161087   192 sched.cpp:336] No credentials provided. Attempting to register without authentication
I0810 14:10:36.162654   192 sched.cpp:703] Framework registered with 4fdf49cd-d2cf-4c31-8204-d71f4ea03c15-0064
Exception in thread "Thread-38" java.lang.NoSuchMethodError: java.util.Map.getOrDefault(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
    at org.apache.spark.scheduler.cluster.mesos.MesosSchedulerUtils$class.passKerberosConf(MesosSchedulerUtils.scala:345)
    at org.apache.spark.scheduler.cluster.mesos.CoarseMesosSchedulerBackend.passKerberosConf(CoarseMesosSchedulerBackend.scala:57)
    at org.apache.spark.scheduler.cluster.mesos.CoarseMesosSchedulerBackend.createCommand(CoarseMesosSchedulerBackend.scala:187)
    at org.apache.spark.scheduler.cluster.mesos.CoarseMesosSchedulerBackend$$anonfun$buildMesosTasks$1.apply(CoarseMesosSchedulerBackend.scala:418)
    at org.apache.spark.scheduler.cluster.mesos.CoarseMesosSchedulerBackend$$anonfun$buildMesosTasks$1.apply(CoarseMesosSchedulerBackend.scala:394)
    at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
    at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
    at org.apache.spark.scheduler.cluster.mesos.CoarseMesosSchedulerBackend.buildMesosTasks(CoarseMesosSchedulerBackend.scala:394)
    at org.apache.spark.scheduler.cluster.mesos.CoarseMesosSchedulerBackend.handleMatchedOffers(CoarseMesosSchedulerBackend.scala:338)
    at org.apache.spark.scheduler.cluster.mesos.CoarseMesosSchedulerBackend.resourceOffers(CoarseMesosSchedulerBackend.scala:310)
I0810 14:10:36.193603   192 sched.cpp:1937] Asked to abort the driver
I0810 14:10:36.193646   192 sched.cpp:1173] Aborting framework '4fdf49cd-d2cf-4c31-8204-d71f4ea03c15-0064'

非常感谢。

Zeppelin Spark无法连接Mesos DC / OS

0 个答案: