pySpark,如何关闭日志消息程序

时间:2018-07-02 10:13:03

标签: pyspark

我正在运行一个交互式pySpark会话,并且正在通知成千上万的此类消息

12: message written to socket.
90: took mesage out of queue: {"DATA":["application_1530518053043_0011","5","2","wn6-mtspar.ec1yjr0ydu0ebmgq53xysfleyg.fx.internal.cloudapp.net","2018-07-02T10:12:27.515+0000","",""],"SOURCE":"SparkExecutorEvents","TAG":"spark-metrics-d65d4762-8dc9-4c9d-a65f-a6af134c168c"}
90: message written to socket.

这使我无法理解代码的进度。

我尝试使用spark.sparkContext.setLogLevel('WARN'),但是消息不会停止出现在屏幕上。

0 个答案:

没有答案