Pyspark / Pyspark内核在Jupyter Notebook中不起作用

Question

已安装内核：

 $jupyter-kernelspec list


Available kernels:
  apache_toree_scala    /usr/local/share/jupyter/kernels/apache_toree_scala
  apache_toree_sql      /usr/local/share/jupyter/kernels/apache_toree_sql
  pyspark3kernel        /usr/local/share/jupyter/kernels/pyspark3kernel
  pysparkkernel         /usr/local/share/jupyter/kernels/pysparkkernel
  python3               /usr/local/share/jupyter/kernels/python3
  sparkkernel           /usr/local/share/jupyter/kernels/sparkkernel
  sparkrkernel          /usr/local/share/jupyter/kernels/sparkrkernel

已创建一个新笔记本，但失败

The code failed because of a fatal error:
    Error sending http request and maximum retry encountered..

jupyter控制台中没有[错误]消息

Answer 1

如果您使用magicspark连接Jupiter笔记本，则还应该启动Livy，这是magicspark用于与Spark集群通信的API服务。

1. 从Apache Livy下载Livy并解压缩
2. 检查是否设置了SPARK_HOME环境，如果没有，请设置为您的Spark安装目录
3. 通过<livy_home>/bin/livy-server在shell /命令行中运行Livy服务器

现在回到笔记本，您应该能够在单元格中运行Spark代码。