Question

我有一个invertIndex的代码如下。但是我对它并不太满意，并且想知道如何使它变得更加紧凑和pythonic

class invertedIndex(object):


  def __init__(self,docs):
     self.docs,self.termList,self.docLists=docs,[],[]

     for index,doc in enumerate(docs):

        for term in doc.split(" "):
            if term in self.termList:
                i=self.termList.index(term)
                if index not in self.docLists[i]:
                    self.docLists[i].append(index)

            else:
                self.termList.append(term)
                self.docLists.append([index])  

  def search(self,term):
        try:
            i=self.termList.index(term)
            return self.docLists[i]
        except:
            return "No results"





docs=["new home sales top forecasts june june june",
                     "home sales rise in july june",
                     "increase in home sales in july",
                     "july new home sales rise"]

i=invertedIndex(docs)
print invertedIndex.search("sales")

Answer 1

将doc标记存储在Python set中，并使用dict引用每个术语的“doc set”。

from collections import defaultdict

class invertedIndex(object):

  def __init__(self,docs):
      self.docSets = defaultdict(set)
      for index, doc in enumerate(docs):
          for term in doc.split():
              self.docSets[term].add(index)

  def search(self,term):
        return self.docSets[term]

docs=["new home sales top forecasts june june june",
                     "home sales rise in july june",
                     "increase in home sales in july",
                     "july new home sales rise"]

i=invertedIndex(docs)
print i.search("sales") # outputs: set([0, 1, 2, 3])

set有点像列表，但是无序，不能包含重复的条目。

defaultdict基本上是dict，当没有数据可用时（默认情况下为空集），它具有默认类型。

Answer 2

这个解决方案几乎与@Peter Gibson相同。在此版本中，索引是数据，没有委托的 docSets 对象。这使得代码略短且更清晰。

代码还保留了文档的原始顺序......这是一种错误，我更喜欢彼得的set()实现。

另请注意，对不存在的术语（如ix['garbage']）的引用会隐式修改索引。如果唯一的API是search，这很好，但这种情况值得注意。

源

class InvertedIndex(dict):
    def __init__(self, docs):
        self.docs = docs

        for doc_index,doc in enumerate(docs):
            for term in doc.split(" "):
                self[term].append(doc_index)

    def __missing__(self, term):
        # operate like defaultdict(list)
        self[term] = []
        return self[term]

    def search(self, term):
        return self.get(term) or 'No results'


docs=["new home sales top forecasts june june june",
      "home sales rise in july june",
      "increase in home sales in july",
      "july new home sales rise",
      'beer',
      ]

ix = InvertedIndex(docs)
print ix.__dict__
print
print 'sales:',ix.search("sales")
print 'whiskey:', ix.search('whiskey')
print 'beer:', ix.search('beer')

print '\nTEST OF KEY SETTING'
print ix['garbage']
print 'garbage' in ix
print ix.search('garbage')

输出

{'docs': ['new home sales top forecasts june june june', 'home sales rise in july june', 'increase in home sales in july', 'july new home sales rise', 'beer']}

sales: [0, 1, 2, 3]
whiskey: No results
beer: [4]

TEST OF KEY SETTING
[]
True
No results

如何使基本的倒排索引程序更加pythonic

2 个答案:

源

输出