使用R data.table计算所有变量组合和df的不同计数

时间:2018-12-21 17:43:15

标签: r dplyr data.table purrr

如何使用data.table在所有可能的m中运行comboGeneral以获取所有可能的变量组合?然后,如何使用这些变量组合来计算所有子集中的数据帧的非重复计数?

这是purrr和dplyr版本。我需要使用data.table进行nms和计数。

library(data.table); library(dplyr); library(magrittr); library(RcppAlgos); library(purrr)

num_m <- seq_len(ncol(mtcars))
nam_list <- names(mtcars)

nms <- map(num_m, ~comboGeneral(nam_list, m = .x, FUN = c)) %>% unlist(recursive = FALSE)

counts <- map_dbl(nms, ~(mtcars %>% select(.x) %>% n_distinct()))

1 个答案:

答案 0 :(得分:2)

不清楚如何通过专门使用Thenable来完成第一部分。 get: function(target, prop) { if (prop === 'then') return null; // I'm not a Thenable // ...the rest of my logic } 来自data.table,因此我认为它的优化程度很高... comboGeneral中的RccpAlgos是替代方案(combn确实不是这样任何实现...):

base

有了这个,data.table中有几种方法:

nms = unlist(lapply(num_m, combn, x = nam_list, simplify = FALSE), recursive = FALSE)

data.table

mtcars = as.data.table(mtcars)
counts = sapply(nms, uniqueN, x = mtcars)

第一个选项似乎不仅最简洁,而且最有效:

sapply(nms, function(nm) nrow(mtcars[ , TRUE, keyby = nm]))

关于加快第一步,您可以通过降低sapply(nms, function(nm) nrow(unique(mtcars, by = nm))) 的糖并使用原始的library(microbenchmark) microbenchmark(times = 100L, map_dbl(nms, ~(mtcars %>% select(.x) %>% n_distinct())), sapply(nms, uniqueN, x = mtcars), sapply(nms, function(nm) nrow(mtcars[ , TRUE, keyby = nm])), sapply(nms, function(nm) nrow(unique(mtcars, by = nm)))) # Unit: milliseconds # expr min lq # map_dbl(nms, ~(mtcars %>% select(.x) %>% n_distinct())) 2246.10862 2365.33801 # sapply(nms, uniqueN, x = mtcars) 66.16144 68.95391 # sapply(nms, function(nm) nrow(mtcars[, TRUE, keyby = nm])) 1659.20425 1701.79188 # sapply(nms, function(nm) nrow(unique(mtcars, by = nm))) 102.42203 106.87100 # mean median uq max neval # 2469.50648 2448.44821 2544.00350 3530.6513 100 # 73.28518 71.54861 75.85161 118.5919 100 # 1796.30372 1766.59618 1825.97374 2881.2376 100 # 113.63032 111.28377 118.22441 174.2691 100 来获得大约10%的加速:

map

注意:我们不能使用lapply,因为microbenchmark(times = 1000L, lapply(num_m, combn, x = nam_list, simplify = FALSE), map(num_m, ~comboGeneral(nam_list, m = .x, FUN = c)), lapply(num_m, function(m) comboGeneral(nam_list, m, FUN = c))) # Unit: microseconds # expr min lq # lapply(num_m, combn, x = nam_list, simplify = FALSE) 1718.994 1847.3710 # map(num_m, ~comboGeneral(nam_list, m = .x, FUN = c)) 564.076 629.5120 # lapply(num_m, function(m) comboGeneral(nam_list, m, FUN = c)) 473.135 525.2655 # mean median uq max neval # 2088.7454 1921.8840 2016.0275 7789.501 1000 # 713.8342 661.0455 709.4650 3800.253 1000 # 593.7732 550.2460 583.7005 5190.982 1000 将被解释为lapply(num_m, comboGeneral, v = nam_list, FUN = c)的参数,而不是FUN的参数。