在Python Pandas中以长格式附加列表元素

Question

我有以下数据：

study_id       list_value
1              ['aaa', 'bbb']
1              ['aaa']
1              ['ccc']
2              ['ddd', 'eee', 'aaa']
2              np.NaN
2              ['zzz', 'aaa', 'bbb']

如何将其转换为类似的内容？

study_id       list_value
1              ['aaa', 'bbb', 'ccc']
1              ['aaa', 'bbb', 'ccc']
1              ['aaa', 'bbb', 'ccc']
2              ['aaa', 'bbb', 'ddd', 'eee', 'zzz'] 
2              ['aaa', 'bbb', 'ddd', 'eee', 'zzz'] 
2              ['aaa', 'bbb', 'ddd', 'eee', 'zzz'] # order of list item doesn't matter

Answer 1

defaultdict

from collections import defaultdict

d = defaultdict(set)

for t in df.dropna(subset=['list_value']).itertuples():
    d[t.study_id] |= set(t.list_value)

df.assign(list_value=df.study_id.map(pd.Series(d).apply(sorted)))


   study_id       list_value
0         1        [a, b, c]
1         1        [a, b, c]
2         1        [a, b, c]
3         2  [a, b, d, e, z]
4         2  [a, b, d, e, z]
5         2  [a, b, d, e, z]

np.unique和其他其他技巧

请注意，结果为ndarray

df.assign(
    list_value=df.study_id.map(
        df.set_index('study_id').list_value.dropna().sum(level=0).apply(np.unique)
    )
)

   study_id       list_value
0         1        [a, b, c]
1         1        [a, b, c]
2         1        [a, b, c]
3         2  [a, b, d, e, z]
4         2  [a, b, d, e, z]
5         2  [a, b, d, e, z]

我们需要使用sorted来完成

df.assign(
    list_value=df.study_id.map(
        df.set_index('study_id').list_value.dropna()
          .sum(level=0).apply(np.unique).apply(sorted)
    )
)

Gross Way！

df.assign(
    list_value=df.study_id.map(
        df.list_value.str.join('|').groupby(df.study_id).apply(
            lambda x: sorted(set('|'.join(x.dropna()).split('|')))
        )
    )
)

   study_id       list_value
0         1        [a, b, c]
1         1        [a, b, c]
2         1        [a, b, c]
3         2  [a, b, d, e, z]
4         2  [a, b, d, e, z]
5         2  [a, b, d, e, z]

设置

df = pd.DataFrame(dict(
    study_id=[1, 1, 1, 2, 2, 2],
    list_value=[['a', 'b'], ['a'], ['c'], ['d', 'e', 'a'], np.nan, ['z', 'a', 'b']]
), columns=['study_id', 'list_value'])

Answer 2

带有itertools.chain的

GroupBy.transform
首先，使用列表理解来消除列中的NaN（我知道这很麻烦，但这是最快的方法）。

df['list_value'] = [
    [] if not isinstance(x, list) else x for x in df.list_value
]

接下来，在study_id上进行分组，并在GroupBy.transform内展平您的列表，并使用set提取唯一值。

from itertools import chain

df['list_value'] = df.groupby('study_id').list_value.transform(
    lambda x: [list(set(chain.from_iterable(x)))]
)

作为最后一步，如果您打算改变单个列表项，您可能想要

df['list_value'] = [x[:] for x in df['list_value']]

如果没有，则一个列表中的更改将反映在该组中的所有子列表中。

df
   study_id                 list_value
0         1            [aaa, ccc, bbb]
1         1            [aaa, ccc, bbb]
2         1            [aaa, ccc, bbb]
3         2  [bbb, ddd, eee, aaa, zzz]
4         2  [bbb, ddd, eee, aaa, zzz]
5         2  [bbb, ddd, eee, aaa, zzz]

Answer 3

这是一种手动方式。

import pandas as pd, numpy as np
from itertools import chain

df = pd.DataFrame({'study_id': [1, 1, 1, 2, 2, 2],
                   'list_value': [['aaa', 'bbb',], ['aaa'], ['ccc'],['ddd', 'eee', 'aaa'],
                                  np.nan, ['zzz', 'aaa', 'bbb']]})

counts = df['study_id'].value_counts()

grp = df.dropna(subset=['list_value'])\
        .groupby('study_id')['list_value']\
        .apply(lambda x: sorted(set(chain.from_iterable(x))))\
        .reset_index()

res = pd.concat([pd.concat([grp[grp['study_id'] == x]]*counts[x]) for x in counts.index])\
        .sort_values('study_id')\
        .reset_index(drop=True)

#    study_id                 list_value
# 0         1            [aaa, bbb, ccc]
# 1         1            [aaa, bbb, ccc]
# 2         1            [aaa, bbb, ccc]
# 3         2  [aaa, bbb, ddd, eee, zzz]
# 4         2  [aaa, bbb, ddd, eee, zzz]
# 5         2  [aaa, bbb, ddd, eee, zzz]

Answer 4

使用空列表填充空值，然后使用transform

df.at[df.list_value.isnull().nonzero()[0][0],'list_value']=[]

df.groupby('study_id').list_value.transform(lambda x : [list(set(x.sum()))])
Out[160]: 
0          [b, c, a]
1          [b, c, a]
2          [b, c, a]
3    [b, e, d, z, a]
4    [b, e, d, z, a]
5    [b, e, d, z, a]
Name: list_value, dtype: object