Question

如何summarize a data.table with unreliable data跨多个列？

具体来说，给出

fields <- c("country","language")
dt <- data.table(user=c(rep(3, 5), rep(4, 5)),
                 behavior=c(rep(FALSE,5),rep(TRUE,5)),
                 country=c(rep(1,4),rep(2,6)),
                 language=c(rep(6,6),rep(5,4)),
                 event=1:10, key=c("user",fields))
dt
#     user behavior country language event
#  1:    3    FALSE       1        6     1
#  2:    3    FALSE       1        6     2
#  3:    3    FALSE       1        6     3
#  4:    3    FALSE       1        6     4
#  5:    3    FALSE       2        6     5
#  6:    4     TRUE       2        5     7
#  7:    4     TRUE       2        5     8
#  8:    4     TRUE       2        5     9
#  9:    4     TRUE       2        5    10
# 10:    4     TRUE       2        6     6

我想要

#    user behavior country.name country.support language.name language.support
# 1:    3    FALSE            1             0.8             6              1.0
# 2:    4     TRUE            2             1.0             5              0.8

（此处 x .name是user和 x {{1}最常见的 x 是观察到这个顶部 x 的共享事件）

无需像这样用手.support：

fields

users <- dt[, sum(behavior) > 0, by=user] # have behavior at least once setnames(users, "V1", "behavior") dt.out <- dt[, .N, by=list(user,country) ][, list(country[which.max(N)],max(N)/sum(N)), by=user] setnames(dt.out, c("V1", "V2"), paste0("country",c(".name", ".support"))) users <- users[dt.out] dt.out <- dt[, .N, by=list(user,language) ][, list(language[which.max(N)], max(N)/sum(N)), by=user] setnames(dt.out, c("V1", "V2"), paste0("language",c(".name", ".support"))) users <- users[dt.out] users # user behavior country.name country.support language.name language.support # 1: 3 FALSE 1 0.8 6 1.0 # 2: 4 TRUE 2 1.0 5 0.8的实际数量是5，我希望避免分别为每个字段重复相同的代码，如果我修改fields，则必须编辑此函数。请注意 this 是这个问题的实质内容，支持计算向我解释elsewhere。

与the referenced question一样，我的数据集大约有10 ^ 7行，所以我真的需要一个可扩展的解决方案;如果我可以像fields中那样避免不必要的复制，那也很好。

Answer 1

这会解决您的问题吗？

fields <- c("country","language")
dt <- data.table(user=c(rep(3, 5), rep(4, 5)),
           behavior=c(rep(FALSE,5),rep(TRUE,5)),
           country=c(rep(1,4),rep(2,6)),
           language=c(rep(6,6),rep(5,4)),
           event=1:10, key=c("user",fields))

CalculateSupport <- function(dt, name) {
  x <- dt[, .N, by = eval(paste0('user,', name))]
  setnames(x, name, 'name')
  x <- x[, list(name[which.max(N)], max(N)/sum(N)), by = user]
  setnames(x, c('V1', 'V2'), paste0(name, c(".name", ".support")))
  x
}

users <- dt[, sum(behavior) > 0, by=user] 
setnames(users, "V1", "behavior")

Reduce(function(x, name) x[CalculateSupport(dt, name)], fields, users)

结果

   user behavior country.name country.support language.name language.support
1:    3    FALSE            1             0.8             6              1.0
2:    4     TRUE            2             1.0             5              0.8

P.S。请认真对待里卡多对你问题的评论。所以有很多很乐意帮助你的人，但是你必须得到很好的尊重。

Answer 2

我无法在一个表达式中执行此操作，因为我不确定如何在data.table表达式中重用已创建的字段。它也可能不是最有效的方式。也许这会成为一个很好的起点。

#Find most common country and language for each user
summ.dt<-dt[,list(behavior.summ=sum(behavior)>0,
     country.name=dt[user==.BY[[1]],.N,by=country][N==max(N),country],
     language.name=dt[user==.BY[[1]],.N,by=language][N==max(N),language]),
by=user]

#Get support for each country and language for each user
summ.dt[,c("country.support","language.support"):=list(
     nrow(dt[user==.BY[[1]] & country==country.name])/nrow(dt[user==.BY[[1]]]),
     nrow(dt[user==.BY[[1]] & language==language.name])/nrow(dt[user==.BY[[1]]])
),by=user]

    user behavior.summ country.name language.name country.support language.support
1:    3         FALSE            1             6             0.8              1.0
2:    4          TRUE            2             5             1.0              0.8

在多个列中汇总data.table

2 个答案: