如何在(ana)conda环境中的Jupyter中为Spark Kernel安装Apache Toree?

时间:2016-05-13 16:05:57

标签: apache-spark anaconda jupyter-notebook conda apache-toree

我正在尝试在JupyterSpark环境(我使用conda设置的http://conda.pydata.org/docs/test-drive.html)中安装anaconda distribution - apache toree支持。 我正在尝试将Jupyter Kernel用作SendMessage

这是我在安装Anaconda后所做的事情:

conda create --name jupyter python=3
source activate jupyter
conda install jupyter
pip install --pre toree
jupyter toree install

一切正常,直到我到达最后一行。我得到了

PermissionError: [Errno 13] Permission denied: '/usr/local/share/jupyter'

这引出了一个问题:为什么它甚至会查看该目录?毕竟它应该留在环境中。因此我exectue

jupyter --paths

并获取

config:
    /home/user/.jupyter
    ~/anaconda2/envs/jupyter/etc/jupyter
    /usr/local/etc/jupyter
    /etc/jupyter
data:
    /home/user/.local/share/jupyter
    ~/anaconda2/envs/jupyter/share/jupyter
    /usr/local/share/jupyter
    /usr/share/jupyter
runtime:
    /run/user/1000/jupyter

我不太清楚发生了什么,以及如何继续运行(如果可能的话)conda环境“jupyter”。

2 个答案:

答案 0 :(得分:8)

Jupyter默认尝试将内核安装到系统范围的内核注册表中。您可以传递--user标志,它将使用用户内核目录。有关详细信息,请参阅kernelspec.py。 以下命令将toree内核安装到用户内核

jupyter toree install --user

答案 1 :(得分:2)

您可以使用--help查看所有可用选项:

$ jupyter toree install --help
A Jupyter kernel for talking to spark

Options
-------

Arguments that take values are actually convenience aliases to full
Configurables, whose aliases are listed on the help line. For more information
on full configurables, see '--help-all'.

--user
    Install to the per-user kernel registry
--replace
    Replace any existing kernel spec with this name.
--sys-prefix
    Install to Python's sys.prefix. Useful in conda/virtual environments.
--debug
    set log level to logging.DEBUG (maximize logging output)
--kernel_name= (ToreeInstall.kernel_name)
    Default: 'Apache Toree'
    Install the kernel spec with this name. This is also used as the base of the
    display name in jupyter.
--spark_home= (ToreeInstall.spark_home)
    Default: '/usr/local/spark'
    Specify where the spark files can be found.
--toree_opts= (ToreeInstall.toree_opts)
    Default: ''
    Specify command line arguments for Apache Toree.
--spark_opts= (ToreeInstall.spark_opts)
    Default: ''
    Specify command line arguments to proxy for spark config.
--interpreters= (ToreeInstall.interpreters)
    Default: 'Scala'
    A comma separated list of the interpreters to install. The names of the
    interpreters are case sensitive.
--python_exec= (ToreeInstall.python_exec)
    Default: 'python'
    Specify the python executable. Defaults to "python"
--log-level= (Application.log_level)
    Default: 30
    Choices: (0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL')
    Set the log level by value or name.
--config= (JupyterApp.config_file)
    Default: ''
    Full path of a config file.

To see all available configurables, use `--help-all`

Examples
--------

    jupyter toree install
    jupyter toree install --spark_home=/spark/home/dir
    jupyter toree install --spark_opts='--master=local[4]'
    jupyter toree install --kernel_name=toree_special
    jupyter toree install --toree_opts='--nosparkcontext'
    jupyter toree install --interpreters=PySpark,SQL
    jupyter toree install --python=python

使用jupyter toree install --sys-prefix是conda和venv环境的最佳选择。