Spark报告“驱动程序（IP）分离”错误

Question

错误消息-

19/03/06 03:58:20 ERROR CoarseGrainedExecutorBackend: Driver 19.17.18.20:58632 disassociated! Shutting down.

我什至试图增加驱动程序的内存，但这没有用。桌子很大，但不确定为什么会失败

使用的命令-

spark-submit  --master yarn   --num-executors 72 --executor-memory 100G  --conf spark.yarn.executor.memoryOverhead=8192  --driver-memory 36G etl.py

控制台错误-

19/03/06 04:06:50 INFO ParseDriver: Parsing command: select account_id ,transid from etl_core.ACCOUNT_TRANSACTION
19/03/06 04:06:50 INFO ParseDriver: Parse Completed
19/03/06 04:06:50 INFO SparkContext: Invoking stop() from shutdown hook