Question

我一直在按照本教程为scala安装spark： https://www.tutorialspoint.com/apache_spark/apache_spark_installation.htm

但是，当我尝试运行spark-shell时，我在控制台中收到此错误。

/usr/local/spark/bin/spark-shell: line 57: /usr/local/spark/bin/bin/spark-submit: No such file or directory

我的bashrc看起来像这样：

export PATH = $PATH:/usr/local/spark/bin
export SCALA_HOME=/usr/local/scala/bin
export PYTHONPATH=$SPARK_HOME/python

那么我错了什么？我以前为python安装了spark，但现在我尝试使用scala。火花会使变量混淆吗？感谢。

Answer 1

您正在搜索的路径中有一个bin太多：

/usr/local/spark/bin/bin/spark-submit

应该是

/usr/local/spark/bin/spark-submit

SPARK_HOME在您的情况下应为/usr/local/spark/，而不是/usr/local/spark/bin/，因为现在似乎是这样。

Answer 2

根据@Wilmerton的回答，我在~/.bashrc内提出了以下工作配置：

# Apache Spark stuff
export JAVA_HOME=/usr/lib/jvm/default-java/jre
export SPARK_HOME=/usr/lib/spark
export SCALA_HOME=/usr/local/scala/bin
export PATH=$PATH:${SPARK_HOME}/bin
export PATH=$PATH:$SCALA_HOME

（我使用default-jdk安装了apt-get install default-jdk，aptitude search jdk生成了以下与p状态不同的条目：

i   default-jdk                                        - Standard Java or Java compatible Development Kit
i A default-jdk-headless                               - Standard Java or Java compatible Development Kit (headless)
i A openjdk-8-jdk                                      - OpenJDK Development Kit (JDK)
i A openjdk-8-jdk-headless                             - OpenJDK Development Kit (JDK) (headless)
iBA openjdk-8-jre                                      - OpenJDK Java runtime, using Hotspot JIT
i A openjdk-8-jre-headless                             - OpenJDK Java runtime, using Hotspot JIT (headless)
i   openjdk-9-jdk-headless                             - OpenJDK Development Kit (JDK) (headless)
iB  openjdk-9-jre                                      - OpenJDK Java runtime, using Hotspot JIT
i A openjdk-9-jre-headless                             - OpenJDK Java runtime, using Hotspot JIT (headless)

）

Answer 3

尝试将bashrc中的第二个环境变量更改为此

export PATH=$PATH:/usr/local/scala/bin

除了Path代替SCALA_HOME

之外，我的.bash_profile与您的几乎相同

键入spark-shell时无法找到spark提交

3 个答案: