对data.table中计算的.BY的操作

Question

作为this question的扩展，我希望运行包含.BY变量的计算，该变量本身就是计算的结果。我审查的问题使用的密钥只是访问现有值，但不会转换或汇总现有值。

在这个例子中，我试图为具有利用data.table的函数的二进制分类器生成ROC（因为现有包中的ROC计算非常慢）。在这种情况下，.BY变量是切点，计算是该切点处概率估计的真阳性和假阳性率。

我可以使用中间data.table执行此操作，但我正在寻找更有效的解决方案。这有效：

# dummy example
library(data.table)
dt <- setDT(get(data(GermanCredit, package='caret'))
            )[, `:=`(y = as.integer(Class=='Bad'),
                     Class = NULL)]
model <- glm(y ~ ., family='binomial', data=dt)
dt[,y_est := predict(model, type='response')]

#--- Generate ROC with specified # of cutpoints  ---
# level of resolution of ROC curve -- up to uniqueN(y_est)
res <- 5 

# vector of cutpoints (thresholds for y_est)
cuts <- dt[,.( thresh=quantile(y_est, probs=0:res/res) )]

# at y_est >= each threshold, how many true positive and false positives?
roc <-  cuts[, .( tpr = dt[y_est>=.BY[[1]],sum(y==1)]/dt[,sum(y==1)],
                  fpr = dt[y_est>=.BY[[1]],sum(y==0)]/dt[,sum(y==0)]
                 ), by=thresh]

plot(tpr~fpr,data=roc,type='s') # looks right

但这不起作用：

# this doesn't work, and doesn't have access to the total positives & negatives
dt[, .(tp=sum( (y_est>=.BY[[1]]) & (y==1)  ),
       fp=sum( (y_est>=.BY[[1]]) & (y==0)  ) ),
   keyby=.(thresh= quantile(y_est, probs=0:res/res) )]
# Error in `[.data.table`(dt, , .(tp = sum((y_est >= .BY[[1]]) & (y == 1)),  : 
#   The items in the 'by' or 'keyby' list are length (6).
#   Each must be same length as rows in x or number of rows returned by i (1000).

是否有惯用的data.table（或至少更有效）的方法来执行此操作？

Answer 1

您可以使用非等联接：

dt[.(thresh = quantile(y_est, probs=0:res/res)), on = .(y_est >= thresh),
   .(fp = sum(y == 0), tp = sum(y == 1)), by = .EACHI][,
   lapply(.SD, function(x) x/x[1]), .SDcols = -"y_est"]
#           fp          tp
#1: 1.00000000 1.000000000
#2: 0.72714286 0.970000000
#3: 0.46857143 0.906666667
#4: 0.24142857 0.770000000
#5: 0.08142857 0.476666667
#6: 0.00000000 0.003333333