Question

我是nltk的新手，并希望获得特定单词的搭配（例如＆＃34; man＆＃34;），以便稍后我会按频率过滤它们并按PMI分数对它们进行排序。

这是我的试用代码，用于检索包含＆＃34; man＆＃34;的bigrams，但它返回一个空列表：

>>> text = "hello, yesterday I have seen a man walking. On the other side there was another man yelling \"who are you, man?\""
>>> tokens = word_tokenize(text)
>>> finder = BigramCollocationFinder.from_words(tokens, window_size=5)
>>> filter_man = lambda w: "man" not in w
>>> finder.apply_word_filter(filter_man)
>>> finder.ngram_fd.items()
[(('have', 'seen'), 1), ((',', 'yesterday'), 1), (('on', 'the'), 1), (('I', 'have'), 1), (('of', 'another'), 1), (('walking', 'on'), 1), (('seen', 'a'), 1), (('hello', ','), 1), (('man', 'walking'), 1), (('side', 'of'), 1), (('the', 'opposite'), 1), (('a', 'man'), 1), (('opposite', 'side'), 1), (('another', 'man'), 1), (('yesterday', 'I'), 1)]
>>> finder.ngram_fd.items()
[]
>>>

我做错了什么？

Answer 1

finder = BigramCollocationFinder.from_words(text.split())
word_filter = lambda w1, w2: "man" not in (w1, w2)
finder.apply_ngram_filter(word_filter)

bigram_measures = nltk.collocations.BigramAssocMeasures()
raw_freq_ranking = finder.nbest(bigram_measures.raw_freq, 10) #top-10

nltk：如何获取包含特定单词的bigrams

1 个答案: