使用mesos的Spark集群

Question

当我尝试使用mesos进行Spark集群操作时，工作会失败。

问题（作业日志失败）

ser/mesos_agent/slaves/bf120f55-2e2b-4c50-8ba9-fb667f0266a7-S0/frameworks/bf120f55-2e2b-4c50-8ba9-fb667f0266a7-0001/executors/driver-20190401172856-0001/runs/fe4a8723-5663-4fed-885f-9578524bd069/spark-2.1.0-bin-hadoop2.7.tgz'
I0401 17:30:28.234169  2551 exec.cpp:161] Version: 1.0.0
I0401 17:30:28.235190  2548 exec.cpp:236] Executor registered on agent 
bf120f55-2e2b-4c50-8ba9-fb667f0266a7-S0
JAVA_HOME is not set

这是我的配置

httpserver

使用python简单服务器：8080

http://simpleHttpServer/spark-2.1.0-bin-hadoop2.7.tgz

spark-env.sh

export JAVA_HOME=/home/pduser/jdk
export MESOS_NATIVE_JAVA_LIBRARY=/usr/local/lib/libmesos.so
export SPARK_EXECUTOR_URI=http://192.168.44.31:8000/spark-2.1.0-bin- 
hadoop2.7.tgz 

〜/ .bashrc

JAVA_HOME=/home/pduser/jdk
CLASSPATH=.:$JAVA_HOME/lib/tools.jar
HADOOP_HOME=/home/pduser/hadoop
export JAVA_HOME
export CLASSPATH
export HD_HOME=/home/pduser/hadoop
export HADOOP_PREFIX=$HD_HOME
export HADOOP_COMMON_HOME=$HD_HOME
export HADOOP_HDFS_HOME=$HD_HOME
export HADOOP_MAPRED_HOME=$HD_HOME
export HADOOP_YARN_HOME=$HD_HOME
export HADOOP_CONF_DIR=$HD_HOME/etc/hadoop
export YARN_CONF_DIR=$HD_HOME/etc/hadoop
export SPARK_HOME=/home/pduser/spark
PATH=$PATH:$JAVA_HOME/bin:$HADOOP_PREFIX/bin:$SPARK_HOME/bin:
export PATH

Answer 1

我解决了将JDK版本更改为OpenJDK 1.8的问题。