Spark非常大的任务警告

Question

我在SparkSQL中收到了很多这样的警告。这个警告告诉我什么，我需要关心这个吗？

WARN TaskSetManager: Stage 1186 contains a task of very large size (4691 KB). The maximum recommended task size is 100 KB.

Answer 1

此链接将帮助您： - Spark using python: How to resolve Stage x contains a task of very large size (xxx KB). The maximum recommended task size is 100 KB

请先在互联网上查询一下