Question

是否存在内存高效的Scala多维数组，例如Java？

我试图通过严格的内存限制来解决Hackerrank问题：256mb。在使用（39384,39384）元素创建2D数组时，我的解决方案因内存不足错误而中断：

Array.ofDim[Long](39384,39384)

在scala控制台中也是如此。

java.lang.OutOfMemoryError: Java heap space
        at scala.reflect.ManifestFactory$$anon$9.newArray(Manifest.scala:115)
        at scala.reflect.ManifestFactory$$anon$9.newArray(Manifest.scala:113)
        at scala.Array$.ofDim(Array.scala:222)
        at Solution$.solve(Solution.scala:4)
        at Solution$$anonfun$main$1.apply$mcVI$sp(Solution.scala:41)
        at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160)
        at Solution$.main(Solution.scala:37)
        at Solution.main(Solution.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at scala.reflect.internal.util.ScalaClassLoader$$anonfun$run$1.apply(ScalaClassLoader.scala:68)
        at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
        at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.asContext(ScalaClassLoader.scala:99)
        at scala.reflect.internal.util.ScalaClassLoader$class.run(ScalaClassLoader.scala:68)
        at scala.reflect.internal.util.ScalaClassLoader$URLClassLoader.run(ScalaClassLoader.scala:99)
        at scala.tools.nsc.CommonRunner$class.run(ObjectRunner.scala:22)
        at scala.tools.nsc.ObjectRunner$.run(ObjectRunner.scala:39)
        at scala.tools.nsc.CommonRunner$class.runAndCatch(ObjectRunner.scala:29)
        at scala.tools.nsc.ObjectRunner$.runAndCatch(ObjectRunner.scala:39)
        at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:72)
        at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:94)
        at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:103)
        at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)

Answer 1

Array.ofDim[Long](39384,39384)创建大小为39384 * 39384 * Long = 1551099456 * 8 = 11 Gb的数组，显然大于256 Mb。只需尝试更少尺寸即可查看其工作原理：

scala> Array.ofDim[Long](3,3)
res10: Array[Array[Long]] = Array(Array(0, 0, 0), Array(0, 0, 0), Array(0, 0, 0))

如果您需要对大型几何空间进行一些坐标处理 - 您可以创建Map[Point, Long]个字体，例如Map(Point(39382, 9000) -> 5L, Point(1,0) -> 9L)。

如果你实际上只需要两个数组（每个39384大小） - 那么只需创建两个数组Array.ofDim[Long](39384,2)

P.S。如果您的算法是可扩展的，您也可以使用Apache Spark的多个节点进行计算。

高效多维可变Scala数组

1 个答案: