将CSV标记为一个列表，而不是使用Python单独标记

Question

我想在一个列表而不是一个单独的列表中标记CSV吗？

with open ('train.csv') as file_object:
    for trainline in file_object:
        tokens_train = sent_tokenize(trainline)
        print(tokens_train)

这就是我获得输出的方式：

['2.1 Separated of trains']
['Principle: The method to make the signal is different.']
['2.2 Context']

我希望它们都在一个列表中

['2.1 Separated of trains','Principle: The method to make the signal is different.','2.2 Context']

Answer 1

由于sent_tokenize()返回一个列表，因此每次只需扩展一个起始列表即可。

alltokens = []

with open ('train.csv') as file_object:
    for trainline in file_object:
        tokens_train = sent_tokenize(trainline)
        alltokens.extend(tokens_train)
    print(alltokens)

或具有列表理解：

with open ('train.csv') as file_object:
    alltokens = [token for trainline in file_object for token in sent_tokenize(trainline)]
print(alltokens)

即使sent_tokenize()返回的列表大于1，这两种解决方案也可以使用。

Answer 2

初始化一个空列表

out = []

并在循环内向其中添加项目。

out.append(tokens_train)

也许您也需要修改令牌生成器。