Question

我具有以下脚本，并且在最后一行中，我试图从名为“ response”的列中的字符串中删除停用词。

问题是，不是“有点烦”变成了“烦”，实际上甚至丢了字母-因此，有点烦就会变得烦。因为“ a”是停用词

有人可以建议我吗？

   import pandas as pd
   from textblob import TextBlob
   import numpy as np
   import os
   import nltk
   nltk.download('stopwords')
   from nltk.corpus import stopwords
   stop = stopwords.words('english')

   path = 'Desktop/fanbase2.csv'
   df = pd.read_csv(path, delimiter=',', header='infer', encoding = "ISO-8859-1")
   #remove punctuation
   df['response'] = df.response.str.replace("[^\w\s]", "")
   #make it all lower case
   df['response'] = df.response.apply(lambda x: x.lower())
   #Handle strange character in source
   df['response'] = df.response.str.replace("‰Ûª", "''")

   df['response'] = df['response'].apply(lambda x: [item for item in x if item not in stop])

Answer 1

在列表理解中（最后一行），您要对照停用词检查每个单词，如果该单词不在停用词中，则要返回它。但是，您正在向其传递字符串。您需要拆分字符串以使LC正常工作。

df = pd.DataFrame({'response':['This is one type of response!', 'Though i like this one more', 'and yet what is that?']})

df['response'] = df.response.str.replace("[^\w\s]", "").str.lower()

df['response'] = df['response'].apply(lambda x: [item for item in x.split() if item not in stop])


0    [one, type, response]
1      [though, like, one]
2                    [yet]

如果要以字符串形式返回响应，请将最后一行更改为

df['response'] = df['response'].apply(lambda x: ' '.join([item for item in x.split() if item not in stop]))

0    one type response
1      though like one
2                  yet

从熊猫数据框中删除停用词

1 个答案: