线程“main”中的异常java.lang.IllegalArgumentException:pyspark不支持任何应用程序选项

时间:2017-06-25 06:12:35

标签: pyspark

在pyspark中配置jupyter笔记本时出错了: 在.bashrc文件中:

export PATH="/root/anaconda3/bin:$PATH"
export PYSPARK_DRIVER_PYTHON=jupyter
export PYSPARK_DRIVER_PYTHON_OPTS="notebook --ip=my ip"

当我输入pyspark时,它会显示[C 13:51:52.029 NotebookApp] Running as root is not recommended. Use --allow-root to bypass.

我使用pyspark --allow-root,但它显示:

Exception in thread "main" java.lang.IllegalArgumentException: pyspark does not support any application options.
at org.apache.spark.launcher.CommandBuilderUtils.checkArgument(CommandBuilderUtils.java:241)
at org.apache.spark.launcher.SparkSubmitCommandBuilder.buildPySparkShellCommand(SparkSubmitCommandBuilder.java:290)
at org.apache.spark.launcher.SparkSubmitCommandBuilder.buildCommand(SparkSubmitCommandBuilder.java:147)
at org.apache.spark.launcher.Main.main(Main.java:86)

它出了什么问题?

1 个答案:

答案 0 :(得分:0)

将.bashrc文件替换为:

export PATH="/root/anaconda3/bin:$PATH"
export PYSPARK_DRIVER_PYTHON=jupyter
export PYSPARK_DRIVER_PYTHON_OPTS="notebook --ip=my ip --allow-root"