PySpark WARN消息

Question

运行PySpark代码时如何禁用以下WARN消息：

Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
18/06/08 21:04:55 WARN SparkConf: In Spark 1.0 and later spark.local.dir will be overridden by the value set by the cluster manager (via SPARK_LOCAL_DIRS in mesos/standalone and LOCAL_DIRS in YARN).
18/06/08 21:04:55 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.

我花了一些时间玩log4.properties，但无法确切知道哪个类记录了这些。

Answer 1

将它放在你的spark上下文中：

sc.setLogLevel("INFO")