高效分组到dict

时间:2018-06-14 21:46:46

标签: python pandas

我有一个元组列表:

[('Player1', 'A', 1, 100),
('Player1', 'B', 15, 100),
('Player2', 'A', 7, 100),
('Player2', 'B', 65, 100),
('Global Total', None, 88, 100)]

我希望以下列格式转换为dict:

{
  'Player1': {
              'A': [1, 12.5],
              'B': [15, 18.75],
              'Total': [16, 18.18]
           },
  'Player2': {
              'A': [7, 87.5],
              'B': [65, 81.25],
              'Total': [72, 81.81]
           },
  'Global Total': {
            'A': [8, 100],
            'B': [80, 100]
         }
}

因此每个播放器词典都有本地总值,并且根据它的全局总值,它是百分比。

目前,我这样做:

fixed_vals = {}
for name, status, qtd, prct in data_set: # This is the list of tuples var
  if name in fixed_vals:
    fixed_vals[name].update({status: [qtd, prct]})
  else:
    fixed_vals[name] = {status: [qtd, prct]}

fixed_vals['Global Total']['Total'] = fixed_vals['Global Total'].pop(None)
total_a = 0
for k, v in fixed_vals.items():
  if k != 'Global Total':
    total_a += v['A'][0]

fixed_vals['Global Total']['A'] = [
  total_a, total_a * 100 / fixed_vals['Global Total']['Total'][0]
]

fixed_vals['Global Total']['B'] = [
  fixed_vals['Global Total']['Total'][0] - total_a,
  fixed_vals['Global Total']['Total'][0] - fixed_vals['Global Total']['A'][1]
]

for player, vals in fixed_vals.items():
  if player != 'Global Total':
    vals['A'][1] = vals['A'][0] * 100 / fixed_vals['Global Total']['A'][0]
    vals['B'][1] = fixed_vals['Global Total']['A'][1] - vals['B'][1]

问题在于这不是很灵活,因为我必须做类似的事情,  但有近12个类别(A,B,......)

有更好的方法吗?也许这对大熊猫来说是微不足道的?

编辑以澄清:

每个玩家没有重复的类别,每个玩家都有相同的序列(有些可能有0但类别是唯一的)

4 个答案:

答案 0 :(得分:2)

每个人似乎都被dict解决方案所吸引,但为什么不尝试转换为pandas

import pandas as pd

# given
tuple_list = [('Player1', 'A', 1, 100),
('Player1', 'B', 15, 100),
('Player2', 'A', 7, 100),
('Player2', 'B', 65, 100),
('Global Total', None, 88, 100)]

# make a dataframe
df = pd.DataFrame(tuple_list , columns = ['player', 'game','score', 'pct'])
del df['pct']
df = df[df.player!='Global Total']
df = df.pivot(index='player', columns='game', values='score')
df.columns.name='' 
df.index.name='' 

# just a check 
assert df.to_dict() == {'A': {'Player1': 1, 'Player2': 7}, 
                        'B': {'Player1': 15, 'Player2': 65}}

#         A   B
#player        
#Player1  1  15
#Player2  7  65
print('Obtained dataset:\n', df)

基本上,你需要的只是'df'数据框,其余的你可以 稍后计算并添加,无需将其保存到字典中。

以下是OP请求的更新:

# the sum across columns is this - this was the 'Grand Total' in the dicts
#  A     8
#  B    80
sum_col = df.sum(axis=0)

# lets calculate the share of each player score:
shares = df / df.sum(axis=0) * 100
assert shares.transpose().to_dict() == {'Player1': {'A': 12.5, 'B': 18.75}, 
                                        'Player2': {'A': 87.5, 'B': 81.25}}
# in 'shares' the columns add to 100%:
#         A     B
#player             
#Player1 12.50 18.75
#Player2 87.50 81.25

# lets mix up a dataframe close to original dictionary structure 
mixed_df = pd.concat([df.A, shares.A, df.B, shares.B], axis=1)
totals = mixed_df.sum(axis=0)
totals.name = 'Total'
mixed_df = mixed_df.append(totals.transpose())
mixed_df.columns = ['A', 'A_pct', 'B', 'B_pct']    
print('\nProducing some statistics\n', mixed_df)

答案 1 :(得分:1)

一种解决方案是使用groupby对来自同一玩家的连续玩家得分进行分组

tup = [('Player1', 'A', 1, 100),('Player1', 'B', 15, 100),('Player2', 'A', 7, 100),    ('Player2', 'B', 65, 100),    ('Global Total', None, 88, 100)]`

然后导入我们的groupby

from itertools import groupby

result = dict((name,dict((x[1],x[2:]) for x in values)) for name,values in groupby(tup,lambda x:x[0]))

然后去更新所有总计

for key in result:
    if key == "Global Total": continue # skip this one ...
    # sum up our player scores
    result[key]['total'] = [sum(col) for col in zip(*result[key].values())]

# you can print the results too
print result

# {'Player2': {'A': (7, 100), 'total': [72, 200], 'B': (65, 100)}, 'Player1': {'A': (1, 100), 'total': [16, 200], 'B': (15, 100)}, 'Global Total': {'total': [88, 100], None: (88, 100)}}

注意此解决方案!需要!所有玩家1的得分都在您的元组中组合在一起,并且所有玩家2的得分都被分组等

答案 2 :(得分:0)

A)将代码分解为可管理的块:

from collections import defaultdict
result = defaultdict(dict)
for (cat, sub, num, percent) in input_list:
    result[cat][sub] = [num, percent]

现在我们有一个关于玩家数量的词典,但是唯一有效的百分比是总数而且我们没有全局计数。

from collections import Counter
def build_global(dct):
    keys = Counter()
    for key in dct:
        if key == "Global Total":
            continue
        for sub_key in dct[key]:
            keys[sub_key] += dct[key][sub_key][0]
    for key in keys:
        dct["Global Total"][key] = [keys[key], 100]

build_global(result)现在为每个事件产生有效的全局计数。

最后:

def calc_percent(dct):
    totals = dct["Global Total"]
    for key in dct:
        local_total = 0
        if key == "Global Total":
            continue
        for sub_key in dct[key]:
            local_total += dct[key][sub_key][0]
            dct[key][sub_key][1] = (dct[key][sub_key][0]/float(totals[sub_key][0])) * 100
        dct[key]['Total'] = [local_total, (local_total/float(dct['Global Total'][None][0])) * 100]

calc_percent(result)通过并构建百分比。

结果是:

defaultdict(<type 'dict'>, 
    {'Player2': {'A': [7, 87.5], 'B': [65, 81.25], 'Total': [72, 81.81818181818183]}, 
     'Player1': {'A': [1, 12.5], 'B': [15, 18.75], 'Total': [16, 18.181818181818183]}, 
     'Global Total': {'A': [8, 100], None: [88, 100], 'B': [80, 100]}})

如果您需要完全指定,则可以删除全局总计中的None条目和dict(result),以将defaultdict转换为香草dict 1}}。

答案 3 :(得分:0)

在Python 3.6 +中使用more_itertools中的重新映射工具:

<强>鉴于

for (int i = 0; i < newDt.Rows.Count;i++ )
    {
        SqlCommand command = new SqlCommand("update batch set sold_qty=@soldqty2 where id=@id2",con);
        command.Parameters.AddWithValue("@soldqty2", Convert.ToInt32(newDt.Rows[i]["QTY"]));
        command.Parameters.AddWithValue("@id2", Convert.ToInt32(newDt.Rows[i]["QTY"]) + Convert.ToInt32(newDt.Rows[i]["BATCH NUM"]));


        rexe=command.ExecuteNonQuery();
    }

<强>代码

import copy as cp
import collections as ct

import more_itertools as mit


data = [
    ("Player1", "A", 1, 100),
    ("Player1", "B", 15, 100),
    ("Player2", "A", 7, 100),
    ("Player2", "B", 65, 100),
    ('Global Total', None, 88, 100)
]

# Discard the last entry
data = data[:-1]

# Key functions
kfunc = lambda tup: tup[0]
vfunc = lambda tup: tup[1:]
rfunc = lambda x: {item[0]: [item[1]] for item in x}

<强>详情

第1步 - 构建# Step 1 remapped = mit.map_reduce(data, kfunc, vfunc, rfunc) # Step 2 intermediate = ct.defaultdict(list) for d in remapped.values(): for k, v in d.items(): intermediate[k].extend(v) # Step 3 remapped["Global Total"] = {k: [sum(v)] for k, v in intermediate.items()} final = cp.deepcopy(remapped) for name, d in remapped.items(): for lbl, v in d.items(): stat = (v[0]/remapped["Global Total"][lbl][0]) * 100 final[name][lbl].append(stat) 组的新词典。

这是通过定义指示如何处理键和值的关键函数来完成的。 reduce函数将值处理为子字典。另请参阅docs for more details on more_itertools.map_reduce

remapped

第2步 - 为查找构建>>> remapped defaultdict(None, {'Player1': {'A': [1], 'B': [15]}, 'Player2': {'A': [7], 'B': [65]}}) dict

intermediate

第3步 - 从后面的词典中构建>>> intermediate defaultdict(list, {'A': [1, 7], 'B': [15, 65]}) 词典

final