我有一个熊猫数据框:
import pandas as pd
test = pd.DataFrame({'words':[['foo','bar none','scare','bar','foo'],
['race','bar none','scare'],
['ten','scare','crow bird']]})
我正在尝试获取数据框列中所有列表元素的单词/短语计数。我目前的解决方案是:
allwords = []
for index, row in test.iterrows():
for word in row['words']:
allwords.append(word)
from collections import Counter
pd.Series(Counter(allwords)).sort_values(ascending=False)
这有效,但我想知道是否有更快的解决方案。注意:我没有使用 ' '.join()
,因为我不希望将短语拆分为单个单词。
答案 0 :(得分:3)
尝试使用 Counter
:
import collections
words = test['words'].tolist()
collections.Counter([x for sublist in words for x in sublist])
Counter({'foo': 2,
'bar none': 2,
'scare': 3,
'bar': 1,
'race': 1,
'ten': 1,
'crow bird': 1})
答案 1 :(得分:2)
为了提高性能不要使用 iterrows
:
from collections import Counter
from itertools import chain
a = pd.Series(Counter(chain.from_iterable(test['words']))).sort_values(ascending=False)
print (a)
scare 3
foo 2
bar none 2
bar 1
race 1
ten 1
crow bird 1
dtype: int64
仅熊猫解决方案:
a = pd.Series([y for x in test['words'] for y in x]).value_counts()
print (a)
scare 3
bar none 2
foo 2
bar 1
race 1
crow bird 1
ten 1
dtype: int64
答案 2 :(得分:2)
让我们用 .hstack
试试 .value_counts
:
pd.value_counts(np.hstack(test['words']))
scare 3
foo 2
bar none 2
ten 1
bar 1
crow bird 1
race 1
dtype: int64