Question

我需要帮助将我的长度数据1558810 x 84转换为1558810 x 4784的广泛数据

让我详细解释一下如何以及为何。我的原始数据如下 - 数据有三个主要栏目 -

id  empId   dept
1   a       social
2   a       Hist
3   a       math
4   b       comp
5   a       social
6   b       comp
7   c       math
8   c       Hist
9   b       math
10  a       comp

id是唯一的密钥，用于指示哪一名员工每天去哪所大学的哪个部门。我需要将其转换如下。

id  empId   dept    social  Hist    math    comp
1   a       social  1       0       0       0
2   a       Hist    0       1       0       0
3   a       math    0       0       1       0
4   b       comp    0       0       0       1
5   a       social  1       0       0       0
6   b       comp    0       0       0       1
7   c       math    0       0       1       0
8   c       Hist    0       1       0       0
9   b       math    0       0       1       0
10  a       comp    0       0       0       1

我有两个数据集，一个有49k行，另一个有155万行。对于具有1100个唯一部门值的较小数据集，我在reshape2包中使用dcast来获取所需的数据集（因此，转换后的数据将具有3 + 1100列和49k行）。但是当我在具有4700个唯一部门值的较大数据集上使用相同的函数时，我的R因内存问题而崩溃。我试过varous其他替代品，如xtabs，reshape等，但每次因内存错误而失败。

我现在为此目的使用粗略的FOR循环 -

columns <- unique(ds$dept)

for(i in 1:length(unique(ds$dept))) 
{
  ds[,columns[i]] <- ifelse(ds$dept==columns[i],1,0)
}

但这非常慢，代码现在已经运行了10个小时。是否有任何解决方法，我缺少？

任何建议都会有很大的帮助！

Answer 1

你可以尝试

df$dept <- factor(df$dept, levels=unique(df$dept))
res <- cbind(df,  model.matrix(~ 0+dept, df))
colnames(res) <- gsub("dept(?=[A-Za-z])", "", colnames(res), perl=TRUE)
res
#   id empId   dept social Hist math comp
#1   1     a social      1    0    0    0
#2   2     a   Hist      0    1    0    0
#3   3     a   math      0    0    1    0
#4   4     b   comp      0    0    0    1
#5   5     a social      1    0    0    0
#6   6     b   comp      0    0    0    1
#7   7     c   math      0    0    1    0
#8   8     c   Hist      0    1    0    0
#9   9     b   math      0    0    1    0
#10 10     a   comp      0    0    0    1

或者你可以试试

cbind(df, as.data.frame.matrix(table(df[,c(1,3)])))

或使用data.table

library(data.table)
setDT(df)
dcast.data.table(df, id + empId + dept ~ dept, fun=length)

或使用qdap

library(qdap)
cbind(df, as.wfm(with(df, mtabulate(setNames(dept, id)))))

数据

df <- structure(list(id = 1:10, empId = c("a", "a", "a", "b", "a", 
"b", "c", "c", "b", "a"), dept = c("social", "Hist", "math", 
"comp", "social", "comp", "math", "Hist", "math", "comp")), .Names = c("id", 
"empId", "dept"), class = "data.frame", row.names = c(NA, -10L))

Answer 2

尝试：

> cbind(dd[1:3], dcast(dd, dd$id~dd$dept, length)[-1])
Using dept as value column: use value.var to override.
   id empId   dept comp Hist math social
1   1     a social    0    0    0      1
2   2     a   Hist    0    1    0      0
3   3     a   math    0    0    1      0
4   4     b   comp    1    0    0      0
5   5     a social    0    0    0      1
6   6     b   comp    1    0    0      0
7   7     c   math    0    0    1      0
8   8     c   Hist    0    1    0      0
9   9     b   math    0    0    1      0
10 10     a   comp    1    0    0      0

数据：

> dput(dd)
structure(list(id = 1:10, empId = structure(c(1L, 1L, 1L, 2L, 
1L, 2L, 3L, 3L, 2L, 1L), .Label = c("a", "b", "c"), class = "factor"), 
    dept = structure(c(4L, 2L, 3L, 1L, 4L, 1L, 3L, 2L, 3L, 1L
    ), .Label = c("comp", "Hist", "math", "social"), class = "factor")), .Names = c("id", 
"empId", "dept"), class = "data.frame", row.names = c("1", "2", 
"3", "4", "5", "6", "7", "8", "9", "10"))

在R中将大的长数据转换为宽

2 个答案:

数据