分组计算,仅限于每组的前N行

时间:2016-02-25 23:52:49

标签: r data.table subset

之前我问了一个问题并得到了一个很好的答案,但我需要将它应用于更具体的问题。 DT需要根据X和Y值分为16个扇区。 X和Y变量表示循环并划分数据表的坐标。我已经成功地将这个数据表划分为16个不同的“扇区”,我需要在每个扇区上应用sCalc函数并输出一个数字。我正在寻找一种更快的方法来做到这一点。

如果需要,请参阅此链接以获取说明:Faster way to subset data table instead of a for loop R

library(data.table)
DT <- data.table(X = rep(1:2000, times = 1600), Y = rep(1:1600, each =   2000), Norm =rnorm(1600*2000), Unif = runif(1600*2000))

sCalc <- function(DT) { 
    setkey(DT, Norm) 
    cells <- DT[1:(nrow(DT)*0.02)] 
    nCells <- nrow(DT) 
    sumCell <- sum(cells[,Norm/sqrt(Unif)]) 
    return(sumCell/nCells) 
} 

startstop <- function(width, y = FALSE) {
    startend <- width - (width/4 - 1)
    start <- round(seq(0, startend, length.out = 4))
    stop <- round(seq(width/4, width, length.out = 4))
    if  (length(c(start,stop)[anyDuplicated(c(start,stop))]) != 0) {
        dup <- anyDuplicated(c(start,stop))
        stop[which(stop == c(start,stop)[dup])] <- stop[which(stop == c(start,stop)[dup])] - 1
}
    if (y == TRUE) {
        coord <- list(rep(start, each = 4), rep(stop, each = 4))
  } else if (y == FALSE) {
        coord <- list(rep(start, times = 4), rep(stop, times = 4))
  }
  return(coord)
}

sectorCalc <- function(x,y,DT) {
    sector <- numeric(length = 16)
    for (i in 1:length(sector)) {
        sect <- DT[X %between% c(x[[1]][i],x[[2]][i]) & Y %between% c(y[[1]][i],y[[2]][i])]
        sector[i] <- sCalc(sect)
    }
    return(sector)
}

x <- startstop(2000)
y <- startstop(1600, y = TRUE)

sectorLoop <- sectorCalc(x,y,DT)

sectorLoop返回:

-4.729271 -4.769156 -4.974996 -4.931120 -4.777013 -4.644919 -4.958968 -4.663221 -4.771545 -4.909868 -4.821098 -4.795526 -4.846709 -4.931514 -4.875148 -4.847105

一种解决方案是使用cut功能。

DT[, x.sect := cut(DT[, X], seq(0, 2000, by = 500), dig.lab=10)]
DT[, y.sect := cut(DT[, Y], seq(0, 1600, by = 400), dig.lab=10)]
sectorRef <- DT[order(Norm), .(sCalc = sum(Norm[1:(0.02*.N)] / sqrt(Unif[1:(0.02*.N)])  )/(0.02*.N)), by = .(x.sect, y.sect)]
sectorRef <- sectorRef[[3]]

上述解决方案返回一个数据表,其值为:

-4.919447 -4.778576 -4.757455 -4.779086 -4.739814 -4.836497 -4.776635 -4.656748 -4.939441 -4.707901 -4.751791 -4.864481 -4.839134 -4.973294 -4.663360 -5.055344

cor(sectorRef, sectorLoop)

以上回报:0.0726904

1 个答案:

答案 0 :(得分:1)

据我所知,我要解释的第一件事是你可以使用.N告诉你每个by=.(...)组中有多少行。我认为这类似于您的nCells

cells占据每组中前2%的行的位置,可以通过索引[1:(0.02*.N)]在矢量级别完成此操作。假设您想要增加Norm的前2%(这是您从setkey(DT, Norm)获得的顺序,虽然设置一个键不仅仅是排序),您可以调用setkey(DT, Norm)在计算之前,如示例中所示,或者为了使您更清楚自己的行为,可以在计算中使用order(Norm)

sum()部分不会改变,因此等效的第三行是:

DT[order(Norm), 
   .(sCalc = sum( Norm[1:(0.02*.N)] / sqrt(Unif[1:(0.02*.N)]) )/.N), 
   by = .(x.sect, y.sect)]

返回16组的操作:

         x.sect      y.sect       sCalc
 1: (1500,2000]  (800,1200] -0.09380209
 2:  (499,1000]   (399,800] -0.09833151
 3:  (499,1000] (1200,1600] -0.09606350
 4:     (0,499]   (399,800] -0.09623751
 5:     (0,499]  (800,1200] -0.09598717
 6: (1500,2000]     (0,399] -0.09306580
 7: (1000,1500]   (399,800] -0.09669593
 8: (1500,2000]   (399,800] -0.09606388
 9: (1500,2000] (1200,1600] -0.09368166
10:  (499,1000]     (0,399] -0.09611643
11: (1000,1500]     (0,399] -0.09404482
12:     (0,499] (1200,1600] -0.09387951
13: (1000,1500] (1200,1600] -0.10069461
14: (1000,1500]  (800,1200] -0.09825285
15:     (0,499]     (0,399] -0.09890184
16:  (499,1000]  (800,1200] -0.09756506