Question

def boolean_search_and(self, text):



    results = []
    and_tokens = self.tokenize(text)
    tokencount = len(and_tokens)

    term1 = and_tokens[0]
    print ' term 1:', term1

    term2 = and_tokens[1]
    print ' term 2:', term2

    #for term in and_tokens:
    if term1 in self._inverted_index.keys():
        resultlist1 = self._inverted_index[term1]
        print resultlist1
    if term2 in self._inverted_index.keys():
        resultlist2 = self._inverted_index[term2]
        print resultlist2
    #intersection of two sets casted into a list                
    results = list(set(resultlist1) & set(resultlist2)) 
    print 'results:', results

    return str(results)

results = [] and_tokens = self.tokenize(text) tokencount = len(and_tokens) term1 = and_tokens[0] print ' term 1:', term1 term2 = and_tokens[1] print ' term 2:', term2 #for term in and_tokens: if term1 in self._inverted_index.keys(): resultlist1 = self._inverted_index[term1] print resultlist1 if term2 in self._inverted_index.keys(): resultlist2 = self._inverted_index[term2] print resultlist2 #intersection of two sets casted into a list results = list(set(resultlist1) & set(resultlist2)) print 'results:', results return str(results)

此代码适用于两个令牌，例如：text =“Hello World”等等，tokens = ['hello'，'world']。我想将它概括为多个标记，因此文本可以是句子，也可以是整个文本文件。
self._inverted_index是一个将令牌保存为键的字典，值是出现键/令牌的DocID。

你好 - ＆gt; [1,2,5,6]
世界 - ＆gt; [1,3,5,7,8]
结果：
你好和世界 - ＆gt; [1,5]

我想达到以下结果：说，（（你好和计算机）和科学）和世界）

我正致力于为多个单词而不仅仅是两个单词。我开始在python这个mornin'工作，所以我不知道它提供了很多功能。

有什么想法吗？

Answer 1

我想将它概括为多个令牌

def boolean_search_and_multi(self, text):
    and_tokens = self.tokenize(text)
    results = set(self._inverted_index[and_tokens[0]])
    for tok in and_tokens[1:]:
        results.intersection_update(self._inverted_index[tok])
    return list(results)

Answer 2

内置set类型是否适合您？

$ python
Python 2.6.5 (r265:79063, Jun 12 2010, 17:07:01)
[GCC 4.3.4 20090804 (release) 1] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> hello = set([1,2,5,6])
>>> world = set([1,3,5,7,8])
>>> hello & world
set([1, 5])

Python：列表/集合的交集

2 个答案: