Question

我正在尝试将mallet主题模型与LDAvis包一起使用。来自topic.model对象的To do so you must extract a number of parameters：phi，theta，vocab，doc.length和term.frequency。

mallet documentation未提及这些参数。如何使用topic.model和mallet.import()从数据生成的MalletLDA()对象中提取它们？

到目前为止，我已经使用mallet来适应主题模型：

id_numbers <- as.integer(c(1, 2, 3))

comments <- c("words to be used for text mining", "that may or may not be interesting", "but could serve as a good example")

df <- data.frame(id_numbers, comments, stringsAsFactors = F)

# Set up topic model
library(mallet)

stoplist <- c("to", "be", "or")
write.csv(stoplist, file = "example_stoplist.csv")

mallet.instances <- mallet.import(
  as.character(df$id_numbers),
  as.character(df$comments),
  "example_stoplist.csv",
  FALSE,
  token.regexp="[\\p{L}']+")

topic.model <- MalletLDA(num.topics=10)
topic.model$loadDocuments(mallet.instances)
vocabulary <- topic.model$getVocabulary()
word.freqs <- mallet.word.freqs(topic.model)
topic.model$setAlphaOptimization(40, 80) # tweaking optimization interval and burn-in iterations)
topic.model$train(400)

topic.words.m <- mallet.topic.words(topic.model, smoothed=TRUE,
                                normalized=TRUE)
dim(topic.words.m)

vocabulary <- topic.model$getVocabulary() 
colnames(topic.words.m) <- vocabulary 

doc.topics.m <- mallet.doc.topics(topic.model, smoothed=T,
                              normalized=T)


doc.topics.df <- as.data.frame(doc.topics.m)
doc.topics.df <- cbind(id_numbers, doc.topics.df)

doc.topic.means.df <- aggregate(doc.topics.df[, 2:ncol(doc.topics.df)],
                                list(doc.topics.df[,1]),
                                mean)

除此之外，我现在需要为JSON生成LDAvis。我尝试了以下方法：

# LDAvis
library(LDAvis)
phi <- t(mallet.topic.words(topic.model, smoothed = TRUE, normalized = TRUE))
phi.count <- mallet.topic.words(topic.model, smoothed = TRUE, normalized = FALSE)

topic.words <- mallet.topic.words(topic.model, smoothed=TRUE, normalized=TRUE)
topic.counts <- rowSums(topic.words)

topic.proportions <- topic.counts/sum(topic.counts)

vocab <- topic.model$getVocabulary() 

doc.tokens <- data.frame(id=c(1:nrow(doc.topics.m)), tokens=0)
for(i in vocab){
  # Find word if word in text
  matched <- grepl(i, df$comments)
  doc.tokens[matched,2] =doc.tokens[matched,2] +  1
}

createJSON(phi = phi, 
           theta = doc.topics.m, 
           doc.length = doc.tokens, 
           vocab = vocab, 
           term.frequency = apply(phi.count, 1, sum))

但是，这会给我以下错误消息：

Error in createJSON(phi = phi, theta = doc.topics.m, doc.length = doc.tokens,  : 
  Number of rows of phi does not match 
      number of columns of theta; both should be equal to the number of topics 
      in the model.

所以我似乎以错误的方式生成了phi和theta矩阵。

Answer 1

尝试从您创建t()的行中删除矩阵转置函数phi。

RMallet以LDAvis期望的格式返回这些矩阵：主题是文档主题（theta）的列和主题词（phi）的行。有时候翻转其中一个是有意义的，这样行或列总是意味着主题，但不是这里。

从mallet LDA对象获取phi，theta，doc.length，vocab，term.frequency？

1 个答案: