读取Spark数据帧时发生Py4JJavaError

Question

在将pandas datframe转换为spark dataframe时，我面临一个问题。 我在本地模式下运行pyspark 2.4版。

代码：

到目前为止，我还没有遇到任何错误，Pandas数据框已转换为Spark数据框，但是当我尝试检查几条记录时，它抛出了以下错误：

 sparkDF.head(2)


 Py4JJavaError: An error occurred while calling o653.collectToPython.
 : org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 
 in stage 50.0 failed 1 times, most recent failure: Lost task 0.0 in stage 
 50.0 (TID 50, localhost, executor driver): org.apache.spark.SparkException: 
 Python worker failed to connect back.