我有一个大约150万行的pandas数据框。我想查找特定列中特定选定单词(全部已知)的出现次数。这适用于一个单词。
d = df["Content"].str.contains("word").value_counts()
但我想从列表中找出多个已知单词如“word1”,“word2”的出现。 word2也可以是word2或wordtwo,如下所示:
word1 40
word2/wordtwo 120
我如何做到这一点?
答案 0 :(得分:3)
IMO最有效的方法之一就是使用sklearn.feature_extraction.text.CountVectorizer传递一个词汇表(你想要计算的词汇列表)。
演示:
In [21]: text = """
...: I have a pandas data frame with approximately 1.5 million rows. I want to find the number of occurrences of specific, selected words in a certain colu
...: mn. This works for a single word. But I want to find out the occurrences of multiple, known words like "word1", "word2" from a list. Also word2 could
...: be word2 or wordtwo, like so"""
In [22]: df = pd.DataFrame(text.split('. '), columns=['Content'])
In [23]: df
Out[23]:
Content
0 \nI have a pandas data frame with approximatel...
1 I want to find the number of occurrences of sp...
2 This works for a single word
3 But I want to find out the occurrences of mult...
4 Also word2 could be word2 or wordtwo, like so
In [24]: from sklearn.feature_extraction.text import CountVectorizer
In [25]: vocab = ['word', 'words', 'word1', 'word2', 'wordtwo']
In [26]: vect = CountVectorizer(vocabulary=vocab)
In [27]: res = pd.Series(np.ravel((vect.fit_transform(df['Content']).sum(axis=0))),
index=vect.get_feature_names())
In [28]: res
Out[28]:
word 1
words 2
word1 1
word2 3
wordtwo 1
dtype: int64
答案 1 :(得分:3)
你可以像这样创建一个字典:
{w: df["Content"].str.contains(w).sum() for w in words}
假设words
是单词列表。