将句子列表转换为单词标记列表

时间:2018-06-03 17:12:19

标签: python nltk tokenize

我有一个列表如下

data_corpus = ["John likes to watch movies",
 "Mary likes movies too", 
"John also likes to watch football games"]

我想要

['John', 'likes', 'to', 'watch', 'movies', 'Mary', 'likes', 'movies', 'too',
 'John', 'also', 'likes', 'to', 'watch', 'football', 'games']

我做

from nltk.tokenize import word_tokenize
tokenized = [word_tokenize(i) for i in data_corpus]
tokenized

ang获取句子列表而不是单词列表

[['John', 'likes', 'to', 'watch', 'movies'],
 ['Mary', 'likes', 'movies', 'too'],
 ['John', 'also', 'likes', 'to', 'watch', 'football', 'games']]

如何解决?

0 个答案:

没有答案