我有大量并行进程运行。似乎仅仅创建了并行scala Futures会造成内存泄漏。
以下示例代码。根据您的机器设置VM标志，以便在您有多个内核时，它不会在开始时直接转储，“ - Xmx100m -XX：+ HeapDumpOnOutOfMemoryError”。
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Await
import scala.concurrent.duration._
object Bug extends App {
println("Memory leak")
while(true){
Future {
val data = new Array[Byte](1000000*1) // 1 MB
println(".")
}
}
}
几分钟后，此代码将显着减慢。堆转储分析显示数组scala.concurrent.forkjoin.ForkJoinTask []似乎溢出。
以下链接似乎相关，但并未真正提供此问题的任何解决方案：https://issues.scala-lang.org/browse/SI-7336
答案 0 :(得分：7)
这里发生的事情是你以比GC更快的速度分配内存。
在while循环的每次执行中，您都要将任务放到线程池中以分配1MB，您可以非常快速地分配内存。
过了一段时间GC会慢下来。
解决方案？分配内存较慢。
顺便说一句，即使你没有声明数组，它也会以类似的方式运行。当您运行Future时，您正在向线程池的队列提交task，因此内存将以非常快的速度增加。
要亲自查看，您可以运行jstat（http://www.cubrid.org/blog/dev-platform/how-to-monitor-java-garbage-collection/）
jstat -gc $pid 1000
这是我笔记本电脑的样本，看看GCT列（GC操作的总累计时间）是如何增加的
➜ ~ jstat -gc 24504 1000
S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT GCT
35328.0 35840.0 22962.9 0.0 275968.0 0.0 395264.0 383080.0 11904.0 11387.3 1664.0 1545.4 30 1.333 3 2.125 3.458
35328.0 35840.0 22962.9 0.0 275968.0 0.0 395264.0 383080.0 11904.0 11387.3 1664.0 1545.4 30 1.333 3 2.125 3.458
35328.0 35840.0 22962.9 0.0 275968.0 0.0 395264.0 383080.0 11904.0 11387.3 1664.0 1545.4 30 1.333 3 2.125 3.458
36352.0 36864.0 8148.0 0.0 275456.0 0.0 562688.0 533891.1 11904.0 11407.6 1664.0 1548.2 36 1.772 4 4.751 6.524
36352.0 36864.0 8148.0 0.0 275456.0 0.0 562688.0 533891.1 11904.0 11407.6 1664.0 1548.2 36 1.772 4 4.751 6.524
36352.0 36864.0 0.0 0.0 275456.0 143402.5 699392.0 486444.1 11904.0 11407.6 1664.0 1548.2 37 1.772 4 6.867 8.640
32768.0 33792.0 22498.3 11986.9 278528.0 278251.8 699392.0 632212.1 11904.0 11437.5 1664.0 1549.8 47 2.548 4 6.867 9.415
29696.0 31232.0 20498.9 0.0 284160.0 0.0 699392.0 678668.1 11904.0 11448.8 1664.0 1551.3 50 2.768 5 6.867 9.635
29696.0 31232.0 20498.9 0.0 284160.0 0.0 699392.0 678668.1 11904.0 11448.8 1664.0 1551.3 50 2.768 5 6.867 9.635
29696.0 31232.0 20498.9 0.0 284160.0 0.0 699392.0 678668.1 11904.0 11448.8 1664.0 1551.3 50 2.768 5 6.867 9.635
29696.0 31232.0 0.0 0.0 284160.0 283932.1 699392.0 691190.3 11904.0 11448.8 1664.0 1551.3 50 2.768 6 10.452 13.220
29696.0 31232.0 0.0 0.0 284160.0 283932.1 699392.0 691190.3 11904.0 11448.8 1664.0 1551.3 50 2.768 6 10.452 13.220
29696.0 31232.0 0.0 0.0 284160.0 283932.1 699392.0 691190.3 11904.0 11448.8 1664.0 1551.3 50 2.768 6 10.452 13.220
29696.0 31232.0 0.0 0.0 284160.0 283932.1 699392.0 691190.3 11904.0 11448.8 1664.0 1551.3 50 2.768 6 10.452 13.220
29696.0 31232.0 0.0 0.0 284160.0 283932.1 699392.0 691190.3 11904.0 11448.8 1664.0 1551.3 50 2.768 6 10.452 13.220
29696.0 31232.0 0.0 0.0 284160.0 283918.2 699392.0 698585.9 11904.0 11451.8 1664.0 1551.3 50 2.768 7 15.069 17.837
29696.0 31232.0 0.0 0.0 284160.0 283918.2 699392.0 698585.9 11904.0 11451.8 1664.0 1551.3 50 2.768 7 15.069 17.837
29696.0 31232.0 0.0 0.0 284160.0 283918.2 699392.0 698585.9 11904.0 11451.8 1664.0 1551.3 50 2.768 7 15.069 17.837
29696.0 31232.0 0.0 0.0 284160.0 283918.2 699392.0 698585.9 11904.0 11451.8 1664.0 1551.3 50 2.768 7 15.069 17.837
29696.0 31232.0 0.0 0.0 284160.0 16821.1 699392.0 699140.7 11904.0 11452.4 1664.0 1551.3 50 2.768 7 19.562 22.330
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699140.7 11904.0 11452.4 1664.0 1551.3 50 2.768 8 19.562 22.330
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699140.7 11904.0 11452.4 1664.0 1551.3 50 2.768 8 19.562 22.330
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699140.7 11904.0 11452.4 1664.0 1551.3 50 2.768 8 19.562 22.330
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699140.7 11904.0 11452.4 1664.0 1551.3 50 2.768 8 19.562 22.330
29696.0 31232.0 0.0 0.0 284160.0 284029.3 699392.0 699116.2 11904.0 11453.7 1664.0 1551.3 50 2.768 9 24.112 26.880
29696.0 31232.0 0.0 0.0 284160.0 284029.3 699392.0 699116.2 11904.0 11453.7 1664.0 1551.3 50 2.768 9 24.112 26.880
29696.0 31232.0 0.0 0.0 284160.0 284029.3 699392.0 699116.2 11904.0 11453.7 1664.0 1551.3 50 2.768 9 24.112 26.880
29696.0 31232.0 0.0 0.0 284160.0 284029.3 699392.0 699116.2 11904.0 11453.7 1664.0 1551.3 50 2.768 9 24.112 26.880
29696.0 31232.0 0.0 0.0 284160.0 284029.3 699392.0 699116.2 11904.0 11453.7 1664.0 1551.3 50 2.768 9 24.112 26.880
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699092.9 11904.0 11455.9 1664.0 1551.3 50 2.768 10 28.728 31.496
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699092.9 11904.0 11455.9 1664.0 1551.3 50 2.768 10 28.728 31.496
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699092.9 11904.0 11455.9 1664.0 1551.3 50 2.768 10 28.728 31.496
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699092.9 11904.0 11455.9 1664.0 1551.3 50 2.768 10 28.728 31.496
29696.0 31232.0 0.0 0.0 284160.0 176242.5 699392.0 699074.7 11904.0 11455.9 1664.0 1551.3 50 2.768 11 33.550 36.318
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699074.7 11904.0 11455.9 1664.0 1551.3 50 2.768 11 33.550 36.318
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699074.7 11904.0 11455.9 1664.0 1551.3 50 2.768 11 33.550 36.318
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699074.7 11904.0 11455.9 1664.0 1551.3 50 2.768 11 33.550 36.318
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699074.7 11904.0 11455.9 1664.0 1551.3 50 2.768 11 33.550 36.318
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699074.7 11904.0 11455.9 1664.0 1551.3 50 2.768 11 33.550 36.318
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699313.9 11904.0 11458.4 1664.0 1551.3 50 2.768 12 38.797 41.565
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699313.9 11904.0 11458.4 1664.0 1551.3 50 2.768 12 38.797 41.565
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699313.9 11904.0 11458.4 1664.0 1551.3 50 2.768 12 38.797 41.565
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699313.9 11904.0 11458.4 1664.0 1551.3 50 2.768 12 38.797 41.565
29696.0 31232.0 0.0 0.0 284160.0 284160.0 699392.0 699313.9 11904.0 11458.4 1664.0 1551.3 50 2.768 12 38.797 41.565