Question

DT = data.table(x=rep(c("b","a","c"),each=3), y=c(1,3,6), v=1:9)

# Desired output
rbind(cbind(id = "v", DT[x == "a", as.list(quantile(.SD, prob = c(0.05, .5, 0.95), na.rm = T)), by = x, .SDcols = c("v")]),
      cbind(id = "y", DT[x == "a", as.list(quantile(.SD, prob = c(0.05, .5, 0.95), na.rm = T)), by = x, .SDcols = c("y")]),

      cbind(id = "v", DT[x == "b", as.list(quantile(.SD, prob = c(0.05, .5, 0.95), na.rm = T)), by = x, .SDcols = c("v")]),
      cbind(id = "y", DT[x == "b", as.list(quantile(.SD, prob = c(0.05, .5, 0.95), na.rm = T)), by = x, .SDcols = c("y")]),
      cbind(id = "v", DT[x == "c", as.list(quantile(.SD, prob = c(0.05, .5, 0.95), na.rm = T)), by = x, .SDcols = c("v")]),
      cbind(id = "y", DT[x == "c", as.list(quantile(.SD, prob = c(0.05, .5, 0.95), na.rm = T)), by = x, .SDcols = c("y")])
)
#    id x  5% 50% 95%
# 1:  v a 4.1   5 5.9
# 2:  y a 1.2   3 5.7
# 3:  v b 1.1   2 2.9
# 4:  y b 1.2   3 5.7
# 5:  v c 7.1   8 8.9
# 6:  y c 1.2   3 5.7

如何使用data.table（在内存中只有几GB）在一个非常大的数据集上最好地实现上述输出？我已经尝试过了，但这不是我想要的

# not right, want all 3 percentiles on the same row, for x and then y:
out <- DT[ , lapply(.SD, quantile, prob = c(0.05, .5, 0.95), na.rm = T), .SDcols = c("v", "y"), keyby = "x"]
out

然后如何获得上面想要的输出，但是id分布在各列中，因此它变成3 x 6的data.table。例如具有3行的v5％v50％v95％y5％y50％y95％列。

Answer 1

您可以使用melt/dcast来实现：

dcast(melt(out[, p := paste0(c(5, 50, 95), "%")], 
           c("p", "x"), 
           variable.name = "id"), 
      id + x ~ ...)[order(x, id)]
#    id x  5% 50% 95%
# 1:  v a 4.1   5 5.9
# 2:  y a 1.2   3 5.7
# 3:  v b 1.1   2 2.9
# 4:  y b 1.2   3 5.7
# 5:  v c 7.1   8 8.9
# 6:  y c 1.2   3 5.7

另一个没有中间结果的选项；

melt(DT[, v := as.numeric(v)], 
     "x",
     c("v", "y"),
     variable.name = "id")[, as.list(quantile(value, 
                                              prob = c(.05, .5, .95))), 
                           .(x, id)][order(x, id)]
#    x id  5% 50% 95%
# 1: a  v 4.1   5 5.9
# 2: a  y 1.2   3 5.7
# 3: b  v 1.1   2 2.9
# 4: b  y 1.2   3 5.7
# 5: c  v 7.1   8 8.9
# 6: c  y 1.2   3 5.7

注意。我将列v转换为numeric（从int）以避免来自melt的讨厌警告。

计算R数据表中许多概率的分位数，用于多个列

1 个答案: