用＆＃34;超大＆＃34;绘制热图。矩阵

Question

我想画一张热图。
我有100k * 100k方阵（50Gb（csv），数字在右上方，其他由0填充）。

我想问一下＆＃34;我如何用R绘制热图？＆＃34;这个庞大的数据集 我在大型RAM机器上尝试使用此代码。

d = read.table("data.csv", sep=",")
d = as.matrix(d + t(d))
heatmap(d)

我尝试过像heatmap.2这样的库（在gplots中）或其他东西 但他们需要花费很多时间和回忆。

Answer 1

我建议您在绘制热图之前对矩阵进行大量下采样，例如：做每个子矩阵的平均值（由@IaroslavDomin建议）：

# example of big mx 10k x 10 k
bigMx <- matrix(rnorm(10000*10000,mean=0,sd=100),10000,10000)

# here we downsample the big matrix 10k x 10k to 100x100
# by averaging each submatrix
downSampledMx <- matrix(NA,100,100)
subMxSide <- nrow(bigMx)/nrow(downSampledMx)
for(i in 1:nrow(downSampledMx)){
  rowIdxs <- ((subMxSide*(i-1)):(subMxSide*i-1))+1
  for(j in 1:ncol(downSampledMx)){
    colIdxs <- ((subMxSide*(j-1)):(subMxSide*j-1))+1
    downSampledMx[i,j] <- mean(bigMx[rowIdxs,colIdxs])
  }
}

# NA to disable the dendrograms
heatmap(downSampledMx,Rowv=NA,Colv=NA) 

对于你的庞大矩阵肯定需要一段时间来计算downSampledMx，但这应该是可行的。

编辑：

我认为下采样应保留可识别的“宏观模式”，例如：请参阅以下示例：

# create a matrix with some recognizable pattern
set.seed(123)
bigMx <- matrix(rnorm(50*50,mean=0,sd=100),50,50)
diag(bigMx) <- max(bigMx) # set maximum value on the diagonal
# set maximum value on a circle centered on the middle
for(i in 1:nrow(bigMx)){
  for(j in 1:ncol(bigMx)){
    if(abs((i - 25)^2 + (j - 25)^2 - 10^2) <= 16)
      bigMx[i,j] <- max(bigMx)
  }
}

# plot the original heatmap
heatmap(bigMx,Rowv=NA,Colv=NA, main="original")


# function used to down sample
downSample <- function(m,newSize){
  downSampledMx <- matrix(NA,newSize,newSize)
  subMxSide <- nrow(m)/nrow(downSampledMx)
  for(i in 1:nrow(downSampledMx)){
    rowIdxs <- ((subMxSide*(i-1)):(subMxSide*i-1))+1
    for(j in 1:ncol(downSampledMx)){
      colIdxs <- ((subMxSide*(j-1)):(subMxSide*j-1))+1
      downSampledMx[i,j] <- mean(m[rowIdxs,colIdxs])
    }
  }
  return(downSampledMx)
}

# downsample x 2 and plot heatmap
downSampledMx <- downSample(bigMx,25)
heatmap(downSampledMx,Rowv=NA,Colv=NA, main="downsample x 2") 

# downsample x 5 and plot heatmap
downSampledMx <- downSample(bigMx,10)
heatmap(downSampledMx,Rowv=NA,Colv=NA, main="downsample x 5") 

这是3张热图：