大型SpatialPolygonsDataFrame导致内存泄漏

时间:2016-07-21 14:37:16

标签: r memory-leaks geospatial sp

我有一个程序将一个大的SpatialPolygonsDataFrame对象(1.4Gb)加载到内存中执行一些分析,然后尝试删除该对象。但是,使用系统命令free查看系统内存会显示该对象保持不变,直到R会话重置为止。 我可以使用rworldmaprworlxtra包重现内存泄漏,制作一个大型世界地图列表,每个地图都为SpatialPolygonsDataFrame,然后尝试删除它们:

install.packages("sp")
install.packages("rworldmap")
install.packages("rworldxtra")
library(sp)
library(rworldmap)
library(rworldxtra)

these.maps.large <- lapply(1:100, function(x) assign(paste0("a_", x), getMap(resolution = "high")))
these.maps.smaller <- lapply(1:20, function(x) assign(paste0("a_", x), getMap(resolution = "high")))
# This frees the memory
rm(list="these.maps.smaller")
gc(reset=T)
# This fails to free the memory
rm(list="these.maps.large")
gc(reset=T)

修改 以下是在每个阶段之后调用system2(&#34; free&#34;)的输出。

Restarting R session...

> library(sp)
> library(rworldmap)
### Welcome to rworldmap ###
For a short introduction type :      vignette('rworldmap')
> library(rworldxtra)
> system2("free")
              total        used        free      shared  buff/cache   available
Mem:      131987656     1386468   118712292      540008    11888896   129731040
Swap:       4194300     3505464      688836
> these.maps.large <- lapply(1:100, function(x) assign(paste0("a_", x), getMap(resolution = "high")))
> system2("free")
              total        used        free      shared  buff/cache   available
Mem:      131987656     2708040   117390660      540008    11888956   128409404
Swap:       4194300     3505464      688836
> rm(list="these.maps.large")
> gc(reset=T)
         used (Mb) gc trigger  (Mb) max used (Mb)
Ncells 585803 31.3    9601876 512.8   585803 31.3
Vcells 711198  5.5   96623732 737.2   711198  5.5
> system2("free")
              total        used        free      shared  buff/cache   available
Mem:      131987656     2708428   117390424      540008    11888804   128409168
Swap:       4194300     3505464      688836

Restarting R session...

> library(sp)
> library(rworldmap)
### Welcome to rworldmap ###
For a short introduction type :      vignette('rworldmap')
> library(rworldxtra)
> system2("free")
              total        used        free      shared  buff/cache   available
Mem:      131987656     1386696   118711988      540008    11888972   129730744
Swap:       4194300     3505464      688836
> these.maps.smaller <- lapply(1:20, function(x) assign(paste0("a_", x), getMap(resolution = "high")))
> system2("free")
              total        used        free      shared  buff/cache   available
Mem:      131987656     1699628   118399100      540008    11888928   129417836
Swap:       4194300     3505464      688836
> rm(list="these.maps.smaller")
> gc(reset=T)
         used (Mb) gc trigger  (Mb) max used (Mb)
Ncells 702817 37.6    2564361 137.0   702817 37.6
Vcells 966452  7.4   21638748 165.1   966452  7.4
> system2("free")
              total        used        free      shared  buff/cache   available
Mem:      131987656     1699612   118399116      540008    11888928   129417852
Swap:       4194300     3505464      688836

有没有人知道为什么会这样,并且可以在不重置会话的情况下删除其中一个大型sp对象?

R版本3.2.3(2015-12-10) 平台:x86_64-redhat-linux-gnu(64位) 运行于:Scientific Linux 7.2(氮气)

1 个答案:

答案 0 :(得分:2)

您不显示输出。我明白了:

> rm(list="these.maps.smaller")
> gc(reset=T)
            used  (Mb) gc trigger   (Mb)  max used  (Mb)
Ncells   7782803 415.7   14442815  771.4   7782803 415.7
Vcells 113371012 865.0  184235296 1405.7 113371012 865.0
> # This fails to free the memory
> rm(list="these.maps.large")
> gc(reset=T)
         used (Mb) gc trigger   (Mb) max used (Mb)
Ncells 524121   28   11554252  617.1   524121   28
Vcells 649283    5  147388236 1124.5   649283    5

表明删除这些大型地图会释放大部分内存;该值几乎等于加载包后在新会话中获得的值。