Question

我有一个有点奇怪的熊猫群。

我有一个源数据框，其中包含三列：客户，日期和项目。我想添加一个包含“项目历史记录”的新列，该列是该客户在更早（由日期定义）行中所有项目的数组。例如，给出此源数据框：

Customer    Date    Item
Bert    01/01/2019  Bread
Bert    15/01/2019  Cheese
Bert    20/01/2019  Apples
Bert    22/01/2019  Pears
Ernie   01/01/2019  Buzz Lightyear
Ernie   15/01/2019  Shellfish
Ernie   20/01/2019  A pet dog
Ernie   22/01/2019  Yoghurt
Steven  01/01/2019  A golden toilet
Steven  15/01/2019  Dominoes

我要创建此历史记录功能：

Customer    Date    Item    Item History
Bert    01/01/2019  Bread   NaN
Bert    15/01/2019  Cheese  [Bread]
Bert    20/01/2019  Apples  [Bread, Cheese]
Bert    22/01/2019  Pears   [Bread, Cheese, Apples]
Ernie   01/01/2019  Buzz Lightyear  NaN
Ernie   15/01/2019  Shellfish   [Buzz Lightyear]
Ernie   20/01/2019  A pet dog   [Buzz Lightyear, Shellfish]
Ernie   22/01/2019  Yoghurt [Buzz Lightyear, Shellfish, A pet dog]
Steven  01/01/2019  A golden toilet NaN
Steven  15/01/2019  Dominoes    [A golden toilet]

我可以执行以下操作以按日期获取历史记录：

df.groupby(['Customer', 'Date']).agg(lambda x: tuple(x)).applymap(list).reset_index()

因此，如果客户在一天之内购买了多个项目，那么它们都被列在一个数组中，而客户只购买了一个单独存在于其自己的数组中的项目，但是我不知道如何将它们连接起来与前面的行。

Answer 1

对GroupBy.transform使用自定义lambda函数，最后将空列表替换为NaN：

f = lambda x: [x[:i].tolist() for i in range(len(x))]
df['Item History'] = df.groupby('Customer')['Item'].transform(f)

具有列表理解功能的另一种解决方案：

df['Item History'] = [x.Item[:i].tolist() for j, x in df.groupby('Customer') 
                                          for i in range(len(x))]

df.loc[~df['Item History'].astype(bool), 'Item History']= np.nan

print (df)
  Customer        Date             Item  \
0     Bert  01/01/2019            Bread   
1     Bert  15/01/2019           Cheese   
2     Bert  20/01/2019           Apples   
3     Bert  22/01/2019            Pears   
4    Ernie  01/01/2019   Buzz Lightyear   
5    Ernie  15/01/2019        Shellfish   
6    Ernie  20/01/2019        A pet dog   
7    Ernie  22/01/2019          Yoghurt   
8   Steven  01/01/2019  A golden toilet   
9   Steven  15/01/2019         Dominoes   

                             Item History  
0                                     NaN  
1                                 [Bread]  
2                         [Bread, Cheese]  
3                 [Bread, Cheese, Apples]  
4                                     NaN  
5                        [Buzz Lightyear]  
6             [Buzz Lightyear, Shellfish]  
7  [Buzz Lightyear, Shellfish, A pet dog]  
8                                     NaN  
9                       [A golden toilet]

Answer 2

我花了很长时间使用@jezrael的答案，但是由于我拥有的数据集大小最终太慢了。为了改善这一点，我创建了一个执行相同功能的函数：

def buildItemHistoryPy(customers, items):

    output = []
    customer_ix = 0

    for i in range(len(customers)):
        if customers[i] == customers[i-1]:
            output.append(items[customer_ix:i])
        else:
            customer_ix = i
            output.append(items[customer_ix:i])

    return output

df['Item History'] = buildItemHistoryPy(df.CustomerAccountNum.values, df.ItemId.values)

我的意图是将其用作Cython函数的基础（我希望它会更快），但是令我惊讶的是，裸露的python函数本身的速度明显要快得多。我继续前进，无论如何都用Cythonise：

%%cython
import numpy as np
cimport numpy as np

cpdef list buildItemHistoryCy(np.ndarray customers, np.ndarray items):

    cdef list output = []
    cdef int customer_ix = 0

    for i in range(len(customers)):
        if customers[i] == customers[i-1]:
            output.append(items[customer_ix:i])
        else:
            customer_ix = i
            output.append(items[customer_ix:i])

    return output

从本质上讲，结果是两种功能都更快，但是Cython的最佳表现是适中的：

%timeit -n5 df['Item History1'] = [x.ItemID[:i].tolist() for j, x in df.groupby('CustomerAccountNum') for i in range(len(x))]
%timeit -n5 df['Item History2'] = buildItemHistoryPy(df.CustomerAccountNum.values, df.ItemID.values)
%timeit -n5 df['Item History3'] = buildItemHistoryCy(df.CustomerAccountNum.values, df.ItemID.values)

7.46 s ± 346 ms per loop (mean ± std. dev. of 7 runs, 5 loops each)
53.5 ms ± 2.16 ms per loop (mean ± std. dev. of 7 runs, 5 loops each)
23.6 ms ± 2.53 ms per loop (mean ± std. dev. of 7 runs, 5 loops each)

我的要求略有变化，因此不再需要清空空列表。如果是这样，则必须更改函数，以便您改为附加items[customer_ix:i].tolist()。

连接熊猫数据框中前几行的值

2 个答案: