Question

我正在使用PySpark和python 3，我刚刚用命令

下载了pyspark

conda install pyspark

并且成功下载，我在Jupyter Notebook中导入了该文件，然后我尝试运行以下代码

sc1 = sp.SparkContext.getOrCreate()

我收到以下异常

Exception                                 Traceback (most recent call last)
<ipython-input-9-a4daca8dcb9d> in <module>()
----> 1 sc1 = sp.SparkContext.getOrCreate()

~\Anaconda3\lib\site-packages\pyspark\context.py in getOrCreate(cls, conf)
    332         with SparkContext._lock:
    333             if SparkContext._active_spark_context is None:
--> 334                 SparkContext(conf=conf or SparkConf())
    335             return SparkContext._active_spark_context
    336 

~\Anaconda3\lib\site-packages\pyspark\context.py in __init__(self, master, appName, sparkHome, pyFiles, environment, batchSize, serializer, conf, gateway, jsc, profiler_cls)
    113         """
    114         self._callsite = first_spark_call() or CallSite(None, None, None)
--> 115         SparkContext._ensure_initialized(self, gateway=gateway, conf=conf)
    116         try:
    117             self._do_init(master, appName, sparkHome, pyFiles, environment, batchSize, serializer,

~\Anaconda3\lib\site-packages\pyspark\context.py in _ensure_initialized(cls, instance, gateway, conf)
    281         with SparkContext._lock:
    282             if not SparkContext._gateway:
--> 283                 SparkContext._gateway = gateway or launch_gateway(conf)
    284                 SparkContext._jvm = SparkContext._gateway.jvm
    285 

~\Anaconda3\lib\site-packages\pyspark\java_gateway.py in launch_gateway(conf)
     93                 callback_socket.close()
     94         if gateway_port is None:
---> 95             raise Exception("Java gateway process exited before sending the driver its port number")
     96 
     97         # In Windows, ensure the Java child processes do not linger after Python has exited.

Exception: Java gateway process exited before sending the driver its port number

我尝试了一些解决方案，但没有一个能够解决问题。有什么建议??

Answer 1

这里为Windows用户提供了解决方案Solution，如何安装，一步一步的指导，我跟着它，它对我有用，试试这个。

Answer 2

Windows中的问题是使用JAVA_HOME目录路径，该路径不能包含“C：\ Program Files”中的任何空格。我在“C：\ Java”目录中重新安装了Java，问题就消失了。

PySpark python错误：异常：在向驱动程序发送其端口号之前退出Java网关进程

2 个答案: