TypeError：'WordListCorpusReader'对象在使用nltk.classify.apply_features时没有属性'__getitem__'

Question

我按照本教程在this site上学习NaiveBayes。我的代码是：

from nltk.corpus import names
from nltk.classify import apply_features

def gender_features(word):
  return {'last_letter': word[-1]}

labeled_names = ([(name, 'male') for name in names.words('male.txt')] +
[(name, 'female') for name in names.words('female.txt')])

feature_sets = [(gender_features(n), gender) for (n, gender) in labeled_names]

#train_set, test_set = feature_sets[500:], feature_sets[:500]
train_set = apply_features(gender_features, names[500:])
test_set = apply_features(gender_features, names[:500])

classifier = NaiveBayesClassifier.train(train_set)

print classifier.classify(gender_features('Neo'))

将train_set与out apply_features一起使用可以正常工作。任何人都知道如何解决它？谢谢。

Answer 1

首先，我认为http://www.nltk.org/book/ch06.html

教程中存在拼写错误

wordlist语料库无法像列表一样访问。

>>> from nltk.corpus import names
>>> names[:5]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'LazyCorpusLoader' object has no attribute '__getitem__'
>>> names.words()[:5]
[u'Abagael', u'Abagail', u'Abbe', u'Abbey', u'Abbi']

接下来请看apply_features做什么（https://github.com/nltk/nltk/blob/develop/nltk/classify/util.py#L28）。

基本上，给定[('input_1', 'label_1'), ...('input_N', 'label_N')]的元组列表，它会返回[(feature_func(tok), label) for (tok, label) in toks]。 E.g。

# To get the input list of tuples for apply_features, we do this:
>>> [(word,'female') for word in names.words('female.txt')[:10]]
[(u'Abagael', 'female'), (u'Abagail', 'female'), (u'Abbe', 'female'), (u'Abbey', 'female'), (u'Abbi', 'female'), (u'Abbie', 'female'), (u'Abby', 'female'), (u'Abigael', 'female'), (u'Abigail', 'female'), (u'Abigale', 'female')]

# Let's get 250 from female and 250 from male names.
>>> train_female = [(word,'female') for word in names.words('female.txt')[:250]] 
>>> train_male = [(word,'male') for word in names.words('male.txt')[:250]]
>>> train_data = train_female + train_male
>>> apply_features(gender_features, train_data)
[({'last_letter': u'l'}, 'female'), ({'last_letter': u'l'}, 'female'), ...]

让Naivebayes在NLTK中使用名称语料库的完整代码：

from nltk.corpus import names
from nltk.classify import apply_features, NaiveBayesClassifier

def gender_features(word):
    return {'last_letter': word[-1]}


train_female = [(word,'female') for word in names.words('female.txt')[:250]] 
train_male = [(word,'male') for word in names.words('male.txt')[:250]]
train_data = train_female + train_male
train_set = apply_features(gender_features, train_data)

# Do like wise for the test set.
'''
test_female = [(word,'female') for word in names.words('female.txt')[250:]]
test_male = [(word,'male') for word in names.words('male.txt')[250:]] 
test_data = test_female + test_male
test_set = apply_features(gender_features, test_data)
'''

classifier = NaiveBayesClassifier.train(train_set)
print classifier.classify(gender_features('Neo'))

[OUT]：

'male'