Question

我有一个包含字符串和浮点列表的数据框，可以说

                        Names                    Prob
           [Anne, Mike, Anne]      [10.0, 10.0, 80.0]
   [Sophie, Andy, Vera, Kate]  [30.0, 4.5, 5.5, 60.0]
         [Josh, Anne, Sophie]            [51, 24, 25]

我想做的是循环遍历Names，如果预定义组中包含名称，请重新标记它，然后汇总Prob中的相应数字。

例如，如果team1 = ['Anne', 'Mike', 'Sophie']我要结束：

                          Names                    Prob
                     [Team_One]                 [100.0]
   [Andy, Kate, Team_One, Vera]  [4.5, 60.0, 30.0, 5.5]
               [Josh, Team_One]                [51, 49]

我写的是这个，但是我认为这有点荒谬，我正在循环内创建一个临时数据帧，然后进行分组；听起来对我来说太过分了，太沉重了。

请问有没有更有效的方法？（如果重要的话，我正在使用Python 3）

import pandas as pd


def pool(df):
    team1 = ['Anne', 'Mike', 'Sophie']

    names = df['Names']
    prob = df['Prob']
    out_names = []
    out_prob = []
    for key, name in enumerate(names):
        # relabel if in team1 otherwise keep it the same
        name = ['Team_One' if x in team1 else x for x in name]

        # make a temp dataframe and group by name
        temp = pd.DataFrame({'name': name, 'prob': prob[key]} )
        temp = temp.groupby('name').sum()

        # make the output
        out_names.append(temp.index.tolist())
        out_prob.append(temp['prob'].tolist())

    df['Names'] = out_names
    df['Prob'] = out_prob
    return df


df = pd.DataFrame({
    'Names':[['Anne', 'Mike', 'Anne'],
             ['Sophie', 'Andy', 'Vera', 'Kate'],
             ['Josh', 'Anne', 'Sophie']
    ],
    'Prob': [[10., 10., 80.],
             [30., 4.5, 5.5, 60.],
             [51, 24, 25]
             ]
})


out = pool(df)
print(out)

谢谢！

Answer 1

使用defaultdict对list中的所有值求和，然后将其转换为元组列表并传递给DataFrame构造函数：

from collections import defaultdict

out = []
for a, b in zipped:
    d = defaultdict(int)
    for x, y in zip(a, b):
        if x in team1:
            d['Team_One'] +=y
        else:
            d[x] = y
    out.append((list(d.keys()), list(d.values())))

df = pd.DataFrame(out, columns=['Names','Prob'])
print (df)
                          Names                    Prob
0                    [Team_One]                 [100.0]
1  [Team_One, Andy, Vera, Kate]  [30.0, 4.5, 5.5, 60.0]
2              [Josh, Team_One]                [51, 49]

如果0中没有Prob值，则解决方案有效：

out = []
for a, b in zipped:
    n, p = [],[]
    tot = 0
    for x, y in zip(a, b):
        if x in team1:
            tot +=y
        else:
            n.append(x)
            p.append(y)
    if tot != 0:    
        p.append(tot)
        n.append('Team_One')

    out.append((n, p))

df = pd.DataFrame(out, columns=['Names','Prob'])
print (df)
                          Names                    Prob
0                    [Team_One]                 [100.0]
1  [Andy, Vera, Kate, Team_One]  [4.5, 5.5, 60.0, 30.0]
2              [Josh, Team_One]                [51, 49]

在熊猫中，使用列表的列的速度很慢，因此最好是先展平列表：

from itertools import chain

lens = [len(x) for x in df['Names']]
df = pd.DataFrame({
    'row' : np.arange(len(df)).repeat(lens),
    'Names' : list(chain.from_iterable(df['Names'].tolist())), 
    'Prob' : list(chain.from_iterable(df['Prob'].tolist()))
})

然后用isin替换值并最后汇总sum：

team1 = ['Anne', 'Mike', 'Sophie']
df.loc[df['Names'].isin(team1), 'Names'] = 'Team_One'

df = df.groupby(['row','Names'], as_index=False, sort=False)['Prob'].sum()
print (df)
   row     Names   Prob
0    0  Team_One  100.0
1    1  Team_One   30.0
2    1      Andy    4.5
3    1      Vera    5.5
4    1      Kate   60.0
5    2      Josh   51.0
6    2  Team_One   49.0

Answer 2

似乎无法解决创建新列表来替换旧列表的问题，因为从原始列表中删除项目的成本太高。我认为这可能是解决名称和问题的可行解决方案，如果名称不在team1中，则将名称和问题附加到新列表中。如果名称在team1中，则不要添加该名称，而应保留team1名称遇到的概率之和。如果在遍历行的每个名称之后该总和不为零，则表明至少找到了一个team1成员（假设所有概率均为正数，如果为true，则为idk）。然后，最后，我们将'Team_One'作为名称和概率总和追加到概率列表（如果总和非零），并用这些新创建的列表替换数据框的列表。

def pool(df):
    # Set of team1 names for faster look up than a list
    team1 = {'Anne', 'Mike', 'Sophie'}

    for i, names in enumerate(df['Names']):
        # iterating through every row and initializing new lists to replace the name/prob lists
        new_names = []
        new_probs = []
        team1_prob = 0
        for name, prob in zip(names, df['Probs'][i]):
            # iterating through every name/prob pair.
            if name not in team1:
                # add the pair to the new lists if not in team1
                new_names.append(name)
                new_probs.append(prob)
            else:
                # keep a sum of probs for all team1 members found, but don't append their name
                team1_prob += prob
        if team1_prob != 0:
            # assuming all probs are positive, thus if any team1 member was found, team1_prob must be nonzero
            new_names.append('Team_One')
            new_probs.append(team1_prob)
        # replace lists in the original df
        df['Names'][i] = new_names
        df['Prob'][i] = new_probs

    return df

使用另一个列表中的键聚合列表

2 个答案: