我正在使用示例here对文本数据执行LDA:
我的问题是:
我如何知道哪些文件对应哪个主题?
换句话说,例如,谈论主题1的文件是什么?
以下是我的步骤:
n_features = 1000
n_topics = 8
n_top_words = 20
我逐行阅读我的文本文件:
with open('dataset.txt', 'r') as data_file:
input_lines = [line.strip() for line in data_file.readlines()]
mydata = [line for line in input_lines]
打印主题的功能:
def print_top_words(model, feature_names, n_top_words):
for topic_idx, topic in enumerate(model.components_):
print("Topic #%d:" % topic_idx)
print(" ".join([feature_names[i]
for i in topic.argsort()[:-n_top_words - 1:-1]]))
print()
对数据进行矢量化:
tf_vectorizer = CountVectorizer(max_df=0.95, min_df=2, token_pattern='\\b\\w{2,}\\w+\\b',
max_features=n_features,
stop_words='english')
tf = tf_vectorizer.fit_transform(mydata)
初始化LDA:
lda = LatentDirichletAllocation(n_topics=3, max_iter=5,
learning_method='online',
learning_offset=50.,
random_state=0)
在tf数据上运行LDA:
lda.fit(tf)
使用上述功能打印结果:
print("\nTopics in LDA model:")
tf_feature_names = tf_vectorizer.get_feature_names()
print_top_words(lda, tf_feature_names, n_top_words)
印刷品的输出是:
Topics in LDA model:
Topic #0:
solar road body lamp power battery energy beacon
Topic #1:
skin cosmetic hair extract dermatological aging production active
Topic #2:
cosmetic oil water agent block emulsion ingredients mixture
答案 0 :(得分:11)
您需要对数据进行转换:
doc_topic = lda.transform(tf)
并列出文档及其最高分主题,如下所示:
for n in range(doc_topic.shape[0]):
topic_most_pr = doc_topic[n].argmax()
print("doc: {} topic: {}\n".format(n,topic_most_pr))
答案 1 :(得分:0)
transform方法将Document Word矩阵X作为输入,并返回X的Document topic分布。
因此,如果您在每个文档中调用转换传递,则可以从您感兴趣的主题中查找那些具有较高(足够用于您的目的)单词的文档。