对不起这个问题,但我对“错误解压”的错误感到疯狂。这是代码
FREQ = 3
fourgrams=""
n = 4
tokens = token_text(text) # is a function that tokenize
fourgrams = ngrams(tokens, n)
final_list = [(item,v) for item,v in nltk.FreqDist(fourgrams) if v > FREQ]
print final_list
错误在哪里?非常感谢
答案 0 :(得分:2)
FreqDist
是一个类似字典的对象。迭代它会产生键(不是键值对)。如果要迭代两个键值对,请使用FreqDist.items
或FreqDist.iteritems
:
final_list = [(item,v) for item,v in nltk.FreqDist(fourgrams).items() if v > FREQ]
答案 1 :(得分:1)
看看这个:
from collections import Counter
from nltk.corpus import brown
from nltk.util import ngrams
# Let's take the first 10000 words from the brown corpus
text = brown.words()[:10000]
# Extract the ngrams
bigrams = ngrams(text, 2)
# Alternatively, unstead of a FreqDist, you can simply use collections.Counter
freqdist = Counter(bigrams)
print len(freqdist)
# Gets the top 5 ngrams
top5 = freqdist.most_common()[:5]
print top5
# Limits v > 10
freqdist = {k:v for k,v in freqdist.iteritems() if v > 10}
print len(freqdist)
[OUT]:
7615
[(('of', 'the'), 95), (('.', 'The'), 76), (('in', 'the'), 59), (("''", '.'), 40), ((',', 'the'), 36)]
34