我想在同一管道中使用Gensim和Scikit。
[更新]
语料库是根据经过修饰的标记列表doc.tokens
bowlist = []
for doc in linked_doc_list:
bowlist.append(doc.tokens)
dictionary = corpora.Dictionary(bowlist)
corpus = [dictionary.doc2bow(line) for line in bowlist]
这涉及将Gensim语料库转换为numpy数组,如下所示:
numpy_matrix = gensim.matutils.corpus2dense(package.corpus, num_terms=len(package.dict.token2id))
这似乎有效。 sklearn lda运行:
model = LatentDirichletAllocation(n_components=components,
max_iter=maxiter,
learning_method=learningmethod,
learning_offset=learningoffset,
random_state=randomstate,
verbose=verbose).fit(numpy_matrix)
但是现在,要阅读结果,我需要阅读gensim dict的实际术语(否则,我将陷入毫无意义的特征编号)。
但是,以下代码的结果显然毫无意义。
def filterAndReportResultsLDA(self, model, gensimdict, n_top_words=10):
for topic_idx, topic in enumerate(model.components_):
print("Topic %d:" % (topic_idx))
words = []
for i in topic.argsort()[:-n_top_words - 1:-1]:
words.append(gensimdict[i])
print(words)
示例结果是:
['reporting.', '7:23', 'users?', 'breaking', '5am', 'bell', 'c7n', 'content?', 'functions', 'vi']
有人可以告诉我我在做什么错吗?