NLTK Word令牌化不返回任何内容

Question

我正在尝试标记一个句子，并且我相信代码是正确的，但是没有输出。可能是什么问题呢？这是代码。

import nltk
from nltk.tokenize import word_tokenize
text = word_tokenize("And now for something completely different")
nltk.pos_tag(text)

text = word_tokenize("They refuse to permit us to obtain the refuse permit")
nltk.pos_tag(text)

Answer 1

似乎缺少以下软件包。

1. punkt
2. averaged_perceptron_tagger

注意：您需要首次下载它们。

尝试一下。

import nltk

nltk.download('punkt')
nltk.download('averaged_perceptron_tagger')

from nltk.tokenize import word_tokenize
text = word_tokenize("And now for something completely different")
print(nltk.pos_tag(text))

text = word_tokenize("They refuse to permit us to obtain the refuse permit")
print(nltk.pos_tag(text))

print("----End of execution----")

Try this on IDE