最近将逻辑从left_outer
连接更新为full
外连接,我开始看到此错误:
ERROR util.ResourceLeakDetector: LEAK: ByteBuf.release() was not called before it's garbage-collected. See http://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records:
Exception in thread "main" org.apache.spark.sql.catalyst.errors.package$TreeNodeException: execute, tree:
Exchange SinglePartition
+- *(7) HashAggregate(keys=[], functions=[partial_count(1)], output=[count#945L])
发现了一个类似的旧错误,但我认为现在应该修复它: https://issues.apache.org/jira/browse/SPARK-21181
对如何解决此问题一无所知。