我正在寻找一个用二元组代替一个字符串的单字组的共现矩阵。我指的是以下链接
http://text2vec.org/glove.html
https://tm4ss.github.io/docs/Tutorial_5_Co-occurrence.html#3_statistical_significance
我想创建矩阵并遍历以创建数据集，如下所示
Trem1 Term2 Score
最大的收获是使用双字词遍历句子。任何帮助都会很棒
答案 0 :(得分：1)
只需指定您的二元组并创建共现矩阵。以下是一些（确实）简单的示例。选择1个包装，然后使用该包装进行所有操作。 Quanteda和text2vec都可以使用多个内核/线程。像reshape2::melt(as.matrix(my_cooccurence_matrix))一样，可以通过reshape2 :: melt遍历所得的共现矩阵。
txt <- c("The quick brown fox jumped over the lazy dog.",
"The dog jumped and ate the fox.")
使用Quanteda创建特征共现矩阵：
library(quanteda)
toks <- tokens(char_tolower(txt), remove_punct = TRUE, ngrams = 2)
f <- fcm(toks, context = "document")
Feature co-occurrence matrix of: 14 by 14 features.
14 x 14 sparse Matrix of class "fcm"
features
features the_quick quick_brown brown_fox fox_jumped jumped_over over_the the_lazy lazy_dog the_dog dog_jumped jumped_and and_ate
the_quick 0 1 1 1 1 1 1 1 0 0 0 0
quick_brown 0 0 1 1 1 1 1 1 0 0 0 0
brown_fox 0 0 0 1 1 1 1 1 0 0 0 0
fox_jumped 0 0 0 0 1 1 1 1 0 0 0 0
jumped_over 0 0 0 0 0 1 1 1 0 0 0 0
over_the 0 0 0 0 0 0 1 1 0 0 0 0
the_lazy 0 0 0 0 0 0 0 1 0 0 0 0
lazy_dog 0 0 0 0 0 0 0 0 0 0 0 0
the_dog 0 0 0 0 0 0 0 0 0 1 1 1
dog_jumped 0 0 0 0 0 0 0 0 0 0 1 1
jumped_and 0 0 0 0 0 0 0 0 0 0 0 1
and_ate 0 0 0 0 0 0 0 0 0 0 0 0
ate_the 0 0 0 0 0 0 0 0 0 0 0 0
the_fox 0 0 0 0 0 0 0 0 0 0 0 0
features
features ate_the the_fox
the_quick 0 0
quick_brown 0 0
brown_fox 0 0
fox_jumped 0 0
jumped_over 0 0
over_the 0 0
the_lazy 0 0
lazy_dog 0 0
the_dog 1 1
dog_jumped 1 1
jumped_and 1 1
and_ate 1 1
ate_the 0 1
the_fox 0 0
使用text2vec创建要素共现矩阵：
library(text2vec)
i <- itoken(txt)
v <- create_vocabulary(i, ngram = c(2L, 2L))
vectorizer <- vocab_vectorizer(v)
f2 <- create_tcm(i, vectorizer)
14 sparse Matrix of class "dgTMatrix"
[[ suppressing 14 column names ‘the_lazy’, ‘and_ate’, ‘The_quick’ ... ]]
the_lazy . . . 0.25 1.0 . 0.2 0.3333333 . . 1.0000000 . 0.5000000 .
and_ate . . . . . 1 . . 0.5000000 1.0 . 0.3333333 . 0.5000000
The_quick . . . 0.50 . . 1.0 0.3333333 . . 0.2000000 . 0.2500000 .
brown_fox . . . . 0.2 . 1.0 1.0000000 . . 0.3333333 . 0.5000000 .
lazy_dog. . . . . . . . 0.2500000 . . 0.5000000 . 0.3333333 .
jumped_and . . . . . . . . 0.3333333 0.5 . 0.5000000 . 1.0000000
quick_brown . . . . . . . 0.5000000 . . 0.2500000 . 0.3333333 .
fox_jumped . . . . . . . . . . 0.5000000 . 1.0000000 .
the_fox. . . . . . . . . . 1.0 . 0.2000000 . 0.2500000
ate_the . . . . . . . . . . . 0.2500000 . 0.3333333
over_the . . . . . . . . . . . . 1.0000000 .
The_dog . . . . . . . . . . . . . 1.0000000
jumped_over . . . . . . . . . . . . . .
dog_jumped . . . . . . . . . . . . . .