Question

我有一个大的数据框（100K行，19列）。我需要计算每个月包含5个项目的每种可能组合的案例数。

以下代码适用于小型数据集，但是对于我的完整数据集而言，它花费的时间太长。通过搜索，我怀疑预先分配一个数据帧是关键，但是我不知道该怎么做。

library(dplyr)

Case<-c(1,1,1,2,2,3,4,5,5,6,6,6,7,8,8,8,9,9,9)
Month<- c("Jan","Jan","Jan","Mar","Mar","Sep","Sep","Nov","Nov","Dec","Dec","Dec","Apr","Dec","Dec","Dec","Dec","Dec","Dec")

Fruits<-c("Apple","Orange","Grape","Grape","Orange","Apple","Apple","Orange","Grape","Apple","Orange","Grape","Grape","Apple","Orange","Grape","Apple","Orange","Grape")

df<-data.frame(Case,Month,Fruits)


Patterns <- with(df, do.call(rbind, lapply(unique(Case), function(x){
  y <- subset(df, Case == x )
  Date<-as.character(y$Month[1])
  Fruits <- paste(unique(y$Fruits[order(y$Fruits)]), collapse = ' / ') 
  as.data.frame(unique (cbind(Case = y$Case, Date, Fruits)))
})))

Total<-Patterns %>%
  group_by(Date,Fruits) %>%
  tally()

我得到的结果是可以接受的，但过程耗时太长，并且由于数据集过多，我的内存不足。

Answer 1

我们可以使用dplyr在一个命令中完成所有操作。首先，我们group_by Case和Month将所有Fruits按组粘贴在一起，然后按Month和Fruits分组，我们添加了行数使用tally的每个组。

library(dplyr)
df %>%
   group_by(Case, Month) %>%
   summarise(Fruits = paste(Fruits, collapse = "/")) %>%
   group_by(Month, Fruits) %>%
   tally()
   # OR count()

#  Month Fruits                 n
#  <fct> <chr>              <int>
#1 Apr   Grape                  1
#2 Dec   Apple/Orange/Grape     3
#3 Jan   Apple/Orange/Grape     1
#4 Mar   Grape/Orange           1
#5 Nov   Orange/Grape           1
#6 Sep   Apple                  2

Answer 2

在大型数据集上，data.table比dplyr快得多：

library(data.table)
setDT(df)[, lapply(.SD, toString), by = c("Case","Month")][,.N, by = c("Fruits","Month")]

如何加快结合rbind和lapply的功能？

2 个答案: