我已经有Java,正在将PySpark与Python 3配合使用,我只是使用命令下载了pyspark
conda install pyspark
我将文件导入Jupyter Notebook,然后尝试运行以下代码
# retrieve SparkContext version
from pyspark import SparkContext
sc = SparkContext()
sc.version
我遇到以下异常
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
<ipython-input-1-60e7cbdbe047> in <module>
1 # retrieve SparkContext version
2 from pyspark import SparkContext
----> 3 sc = SparkContext()
4 sc.version
~/anaconda3/lib/python3.7/site-packages/pyspark/context.py in __init__(self, master, appName, sparkHome, pyFiles, environment, batchSize, serializer, conf, gateway, jsc, profiler_cls)
131 " note this option will be removed in Spark 3.0")
132
--> 133 SparkContext._ensure_initialized(self, gateway=gateway, conf=conf)
134 try:
135 self._do_init(master, appName, sparkHome, pyFiles, environment, batchSize, serializer,
~/anaconda3/lib/python3.7/site-packages/pyspark/context.py in _ensure_initialized(cls, instance, gateway, conf)
314 with SparkContext._lock:
315 if not SparkContext._gateway:
--> 316 SparkContext._gateway = gateway or launch_gateway(conf)
317 SparkContext._jvm = SparkContext._gateway.jvm
318
~/anaconda3/lib/python3.7/site-packages/pyspark/java_gateway.py in launch_gateway(conf)
44 :return: a JVM gateway
45 """
---> 46 return _launch_gateway(conf)
47
48
~/anaconda3/lib/python3.7/site-packages/pyspark/java_gateway.py in _launch_gateway(conf, insecure)
106
107 if not os.path.isfile(conn_info_file):
--> 108 raise Exception("Java gateway process exited before sending its port number")
109
110 with open(conn_info_file, "rb") as info:
Exception: Java gateway process exited before sending its port number
我尝试了一些解决方案,但是没有一个起作用。有什么建议吗?