我在python中正在做一个Question Answering项目。我已经有问题和答案文件的矢量以及tfidf的值。但后来我不知道如何在python中计算相似性匹配。
答案 0 :(得分:1)
您可以在两个向量之间使用Euclidean distance,或者使用另一个距离度量(例如Hamming distance)或向量的cross-correlation。
答案 1 :(得分:1)
余弦相似
length_question = .0
length_answer = .0
for word_tfidf in question:
length_question += word_tfidf**2
for word_tfdif in answer:
length_answer += word_tfidf**2
similarity = .0
for word in question:
question_word_tfidf = question[word]
answer_word_tfidf = answer.get(word, 0)
similarity += question_word_tfidf * answer_word_tfidf
similarity /= math.sqrt(length_question * length_answer)
答案 2 :(得分:1)
您可以使用Levenshtein距离,在此处查看:http://en.wikibooks.org/wiki/Algorithm_Implementation/Strings/Levenshtein_distance#Python代码,此处:http://en.wikipedia.org/wiki/Levenshtein_distance用于讨论算法。
以下是从上述链接复制的代码段:
def levenshtein(s1, s2):
if len(s1) < len(s2):
return levenshtein(s2, s1)
if not s1:
return len(s2)
previous_row = xrange(len(s2) + 1)
for i, c1 in enumerate(s1):
current_row = [i + 1]
for j, c2 in enumerate(s2):
insertions = previous_row[j + 1] + 1 # j+1 instead of j since previous_row and current_row are one character longer
deletions = current_row[j] + 1 # than s2
substitutions = previous_row[j] + (c1 != c2)
current_row.append(min(insertions, deletions, substitutions))
previous_row = current_row
return previous_row[-1]