大期文档矩阵/ simple_triplet_matrix的行和? {tm package}

时间:2014-02-20 22:50:44

标签: r text-mining

所以我有一个非常大的术语文档矩阵:

> class(ph.DTM)
[1] "TermDocumentMatrix"    "simple_triplet_matrix"

> ph.DTM
A term-document matrix (109996 terms, 262811 documents)

Non-/sparse entries: 3705693/28904453063
Sparsity           : 100%
Maximal term length: 191 
Weighting          : term frequency (tf)

如何获得每个学期的rowSum(频率)?我试过了:

> apply(ph.DTM, 1, sum)
Error in vector(typeof(x$v), nr * nc) : vector size cannot be NA
In addition: Warning message:
In nr * nc : NAs produced by integer overflow

显然,我知道removeSparseTerms

ph.DTM2 <- removeSparseTerms(ph.DTM, 0.99999)

缩小了尺寸:

> ph.DTM2
A term-document matrix (28842 terms, 262811 documents)

Non-/sparse entries: 3612620/7576382242
Sparsity           : 100%
Maximal term length: 24 
Weighting          : term frequency (tf)

但我仍然不能对它应用任何与矩阵相关的函数:

> as.matrix(ph.DTM2)
Error in vector(typeof(x$v), nr * nc) : vector size cannot be NA
In addition: Warning message:
In nr * nc : NAs produced by integer overflow

我怎样才能在这个对象上得到一个简单的行和?谢谢!

3 个答案:

答案 0 :(得分:22)

好的,经过更多Google游戏之后,我遇到了slam包,它启用了:

ph.DTM3 <- rollup(ph.DTM, 2, na.rm=TRUE, FUN = sum)

哪个有效。

答案 1 :(得分:10)

@badpanda在其中一条评论中提到,slam现在有稀疏数组的row_sumscol_sums函数:

slam::row_sums(dtm, na.rm = T)
slam::col_sums(tdm, na.rm = T)

答案 2 :(得分:3)

我想:

 rowSums(as.matrix(ph.DTM))

也可以。