通过命令行使用jupyter内核运行python模块

时间:2019-11-28 21:40:07

标签: python shell pyspark jupyter

我将内核配置为在jupyter实验室上运行pyspark客户端。

> jupyter kernelspec list

mykernel       /home/mypath/jupyter/kernels/mykernel

内核配置文件(kernel.json)有很多我想重用的参数

> cat kernel.json

{
  "display_name": "mykernel",
  "language": "python",
  "argv": [
    "/home/mypath/environments/python2/bin/python",
    "-m",
    "ipykernel_launcher",
    "-f",
    "{connection_file}"
  ],
  "env": {
    "HADOOP_USER_NAME": "myusername",
    "YARN_CONF_DIR": "/somepath/hadoop/conf",
    "SPARK_HOME": "/somepath/spark/spark-2.0.0",
    "PYTHONPATH": "/somepath/spark/spark-2.0.0/python/:/somepath/spark/spark-2.0.0/python/lib/py4j-0.10.1-src.zip",
    "PYTHONSTARTUP": "/somepath/kernels/pyspark/start.py",
    "PYSPARK_SUBMIT_ARGS": "--alotofargs --moreargs --moreargs"
  }
}

是否可以通过命令行执行类似的操作?

jupyter mykernel run mypysparkmodule.py

非常感谢!

0 个答案:

没有答案