Question

我有以下文件：

id  review
1   "Human machine interface for lab abc computer applications."
2   "A survey of user opinion of computer system response time."
3   "The EPS user interface management system."
4   "System and human system engineering testing of EPS."              
5   "Relation of user perceived response time to error measurement."
6   "The generation of random binary unordered trees."
7   "The intersection graph of paths in trees."
8   "Graph minors IV Widths of trees and well quasi ordering."
9   "Graph minors A survey."
10  "survey is a state of art."

每一行都涉及一份文件。

我将这些文档转换为语料库，我找到每个单词的TFIDF：

from collections import defaultdict
import csv
from sklearn.feature_extraction.text import TfidfVectorizer

reviews = defaultdict(list)
with open("C:/Users/user/workspacePython/Tutorial/data/unlabeledTrainData.tsv", "r") as sentences_file:
    reader = csv.reader(sentences_file, delimiter='\t')
    reader.next()
    for row in reader:
        reviews[row[1]].append(row[1])

for id, review in reviews.iteritems():
    reviews[id] = " ".join(review)


corpus = []
for id, review in sorted(reviews.iteritems(), key=lambda t: id):
    corpus.append(review)

tf = TfidfVectorizer(analyzer='word', ngram_range=(1,1), min_df = 1, stop_words = 'english')
tfidf_matrix = tf.fit_transform(corpus)

我的问题是：如何在tfidf_matrix中获取给定文档（来自上述文件）的相应向量（行）。

谢谢

Answer 1

您有一个文档列表，1到10.在数组索引术语中，这是0到9。变量tfidx_matrix将包含一个稀疏行形式矩阵，由行（表示文档）和它们与语料库中的词汇表的规范化关联（减去英语停用词）组成。

因此，要将稀疏数组转换为更传统的矩阵，您可以尝试

npm_tfidf = tfidf_matrix.todense()
document_1_vector = npm_tfidf[0]
document_2_vector = npm_tfidf[1]
document_3_vector = npm_tfidf[2]
...
document_10_vector = npm_tfidf[9]

有更简单，更好的方法来提取内容，但我认为阻碍你的方式是从稀疏矩阵表示转换，可能很难解开，以及更传统的密集矩阵表示。

另请注意，解释向量将要求您能够提取在过程中提取的词汇 - 这应该是有序（按字母顺序排列的令牌）的形式，并且可以使用以下方式提取：

vocabulary = tfidf_matrix.get_feature_names()

如何获取给定文档的tfidf向量

1 个答案: