如何计算单词的频率并在列表列表中添加单词的相关权重

时间:2017-06-16 07:39:46

标签: python list frequency

我有以下数据

[[4, 'ABC'], [4, 'BCD'], [3, 'CDE'], [3, 'ABC'], [3, 'DEF'], [3, 'BCD'], [3, 'BCD'], [3, 'BCD']]

我需要以下输出

[ABC, 2, 7]
[BCD, 4, 13]
[CDE, 1, 3]
[DEF, 1, 3]

我需要将单词数量计为位置[1],并将位置为[0]的单词的数字相加。结果是

[Word, freq, sum of weight] 

我检查了finding frequencies of pair items in a list of pairsFinding frequency distribution of a list of numbers in python,但他们无法解决我的问题。

我尝试过但没有成功

res = [[4, 'ABC'], [4, 'BCD'], [3, 'CDE'], [3, 'ABC'], [3, 'DEF'], [3, 'BCD'], [3, 'BCD'], [3, 'BCD']]
 d = {}
for freq, label in res:
    if label not in d:
        d[label] = {}
    inner_dict = d[label]
    if freq not in inner_dict:
        inner_dict[freq] = 0
    inner_dict[freq] += freq

print(inner_dict)

5 个答案:

答案 0 :(得分:5)

用熊猫:

import pandas
data = [[4, 'ABC'], [4, 'BCD'], [3, 'CDE'], [3, 'ABC'], [3, 'DEF'], [3, 'BCD'], [3, 'BCD'], [3, 'BCD']]
df = pandas.DataFrame(data, columns=['count', 'word'])
result = df.groupby('word')['count'].agg((len, sum))

结果:

       len sum
word
ABC      2   7
BCD      4  13
CDE      1   3
DEF      1   3

要对结果进行排序,请使用sort_values

result.sort_values(['sum', 'len'])

      len  sum
word
CDE     1    3
DEF     1    3
ABC     2    7
BCD     4   13

答案 1 :(得分:3)

试试这个:

data = [[4, 'ABC'], [4, 'BCD'], [3, 'CDE'], [3, 'ABC'], [3, 'DEF'], [3, 'BCD'], [3, 'BCD'], [3, 'BCD']]

result = {}
for weight, value in data:
    if value not in result:
        result[value] = [1, weight]
    else:
        result[value][0] += 1
        result[value][1] += weight

print(result)

结果:

{'ABC': [2, 7], 'BCD': [4, 13], 'CDE': [1, 3], 'DEF': [1, 3]}

答案 2 :(得分:1)

您只需使用defaultdictlist comprehension

即可
a = [[4, 'ABC'], [4, 'BCD'], [3, 'CDE'], [3, 'ABC'], [3, 'DEF'], [3, 'BCD'], [3, 'BCD'], [3, 'BCD']]
from collections import defaultdict

d = defaultdict(lambda  : 0)
d2 = defaultdict(lambda : 0)
for i in a:
    d[i[1]] +=1
for i in a :
    d2[i[1]] += i[0]

res =    [ [i, d[i], d2[i]] for i in d.keys() ]

输出:

[['CDE', 1, 3], ['DEF', 1, 3], ['BCD', 4, 13], ['ABC', 2, 7]]
编辑:正如@chthonicdaemon所指出的,初始化defaultdict的一种简单方法是传递int以将其初始化为0,如果需要空字符串则为str

答案 3 :(得分:0)

这里有一个功能性的方法:

l = [[4, 'ABC'], [4, 'BCD'], [3, 'CDE'], [3, 'ABC'], [3, 'DEF'], [3, 'BCD'], [3, 'BCD'], [3, 'BCD']]
data = itertools.groupby(l, key=lambda x: x[1]))
[(k, len(x), sum(x)) for k, x in map(lambda (x, y): (x, map(lambda x: x[0], list(y))), data)]
[('ABC', 1, 4), ('BCD', 1, 4), ('CDE', 1, 3), ('ABC', 1, 3), ('DEF', 1, 3), ('BCD', 3, 9)]

答案 4 :(得分:0)

如果您有一个键的值可以将它们附加到列表中,请使用you_dictionary.setdefault(key,[]).append(value)方法。

a = [[4, 'ABC'], [4, 'BCD'], [3, 'CDE'], [3, 'ABC'], [3, 'DEF'], [3, 'BCD'], [3, 'BCD'], [3, 'BCD']]
my_dict = {}

for item in a:
    key,value=item[1],item[0]
    my_dict.setdefault(key,[]).append(value)
print(my_dict)

my_list = []

for k,v in my_dict.items():
    my_list.append([k,len(v),sum(v)])

print(my_list)

输出:

{'BCD': [4, 3, 3, 3], 'DEF': [3], 'CDE': [3], 'ABC': [4, 3]}
[['BCD', 4, 13], ['DEF', 1, 3], ['CDE', 1, 3], ['ABC', 2, 7]]