Question

有时，似乎我使用Python（和Pandas）越多，我理解的越少。所以我很抱歉，如果我在这里看不到树木的木头，但我已经绕圈而行，只是看不出我做错了什么。

基本上，我有一个示例脚本（我想在更大的数据帧上实现）但我无法让它让我满意。

数据框由各种数据类型的列组成。我想将数据帧分组在2列上，然后生成一个新的数据框，其中包含每个组中每个变量的所有唯一值的列表。（最终，我想将列表项连接成一个字符串 - 但这是一个不同的问题。）

我使用的初始脚本是：

import numpy as np
import pandas as pd

def tempFuncAgg(tempVar):
    tempList = set(tempVar.dropna()) # Drop NaNs and create set of unique values
    print(tempList)
    return tempList

# Define dataframe
tempDF = pd.DataFrame({ 'id': [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20],
                        'date': ["02/04/2015 02:34","06/04/2015 12:34","09/04/2015 23:03","12/04/2015 01:00","15/04/2015 07:12","21/04/2015 12:59","29/04/2015 17:33","04/05/2015 10:44","06/05/2015 11:12","10/05/2015 08:52","12/05/2015 14:19","19/05/2015 19:22","27/05/2015 22:31","01/06/2015 11:09","04/06/2015 12:57","10/06/2015 04:00","15/06/2015 03:23","19/06/2015 05:37","23/06/2015 13:41","27/06/2015 15:43"],
                        'gender': ["male","female","female","male","male","female","female",np.nan,"male","male","female","male","female","female","male","female","male","female",np.nan,"male"],
                        'age': ["young","old","old","old","old","old",np.nan,"old","old","young","young","old","young","young","old",np.nan,"old","young",np.nan,np.nan]})

# Groupby based on 2 categorical variables
tempGroupby = tempDF.groupby(['gender','age'])

# Aggregate for each variable in each group using function defined above
dfAgg = tempGroupby.agg(lambda x: tempFuncAgg(x))
print(dfAgg)

此脚本的输出符合预期：一系列包含值集的行和包含返回集的数据帧：

{'09/04/2015 23:03', '21/04/2015 12:59', '06/04/2015 12:34'}
{'01/06/2015 11:09', '12/05/2015 14:19', '27/05/2015 22:31', '19/06/2015 05:37'}
{'15/04/2015 07:12', '19/05/2015 19:22', '06/05/2015 11:12', '04/06/2015 12:57', '15/06/2015 03:23', '12/04/2015 01:00'}
{'02/04/2015 02:34', '10/05/2015 08:52'}
{2, 3, 6}
{18, 11, 13, 14}
{4, 5, 9, 12, 15, 17}
{1, 10}
                                                           date  \
gender age                                                        
female old    set([09/04/2015 23:03, 21/04/2015 12:59, 06/04...   
       young  set([01/06/2015 11:09, 12/05/2015 14:19, 27/05...   
male   old    set([15/04/2015 07:12, 19/05/2015 19:22, 06/05...   
       young          set([02/04/2015 02:34, 10/05/2015 08:52])   

                                      id  
gender age                                
female old                set([2, 3, 6])  
       young       set([18, 11, 13, 14])  
male   old    set([4, 5, 9, 12, 15, 17])  
       young                set([1, 10])

当我尝试将集转换为列表时，会出现问题。奇怪的是，它产生2个包含相同列表的重复行，但随后出现'ValueError：Function not reduce'错误。

def tempFuncAgg(tempVar):
    tempList = list(set(tempVar.dropna()))   # This is the only difference
    print(tempList)
    return tempList


tempDF = pd.DataFrame({ 'id': [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20],
                        'date': ["02/04/2015 02:34","06/04/2015 12:34","09/04/2015 23:03","12/04/2015 01:00","15/04/2015 07:12","21/04/2015 12:59","29/04/2015 17:33","04/05/2015 10:44","06/05/2015 11:12","10/05/2015 08:52","12/05/2015 14:19","19/05/2015 19:22","27/05/2015 22:31","01/06/2015 11:09","04/06/2015 12:57","10/06/2015 04:00","15/06/2015 03:23","19/06/2015 05:37","23/06/2015 13:41","27/06/2015 15:43"],
                        'gender': ["male","female","female","male","male","female","female",np.nan,"male","male","female","male","female","female","male","female","male","female",np.nan,"male"],
                        'age': ["young","old","old","old","old","old",np.nan,"old","old","young","young","old","young","young","old",np.nan,"old","young",np.nan,np.nan]})

tempGroupby = tempDF.groupby(['gender','age'])

dfAgg = tempGroupby.agg(lambda x: tempFuncAgg(x))
print(dfAgg)

但现在输出是：

['09/04/2015 23:03', '21/04/2015 12:59', '06/04/2015 12:34']
['09/04/2015 23:03', '21/04/2015 12:59', '06/04/2015 12:34']
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
...
ValueError: Function does not reduce

任何帮助解决这个问题的任何帮助都会受到赞赏，如果这是我没有看到的显而易见的事情，我会提前道歉。

EDIT 顺便说一句，将集合转换为元组而不是列表没有问题。

Answer 1

列表有时会在熊猫中出现奇怪的问题。你可以：

使用元组（正如您已经注意到的那样）
如果您确实需要列表，只需在第二个操作中执行此操作：

dfAgg.applymap(lambda x: list(x))

完整示例：

import numpy as np
import pandas as pd

def tempFuncAgg(tempVar):
    tempList = set(tempVar.dropna()) # Drop NaNs and create set of unique values
    print(tempList)
    return tempList

    # Define dataframe
    tempDF = pd.DataFrame({ 'id': [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20],
                            'date': ["02/04/2015 02:34","06/04/2015 12:34","09/04/2015 23:03","12/04/2015 01:00","15/04/2015 07:12","21/04/2015 12:59","29/04/2015 17:33","04/05/2015 10:44","06/05/2015 11:12","10/05/2015 08:52","12/05/2015 14:19","19/05/2015 19:22","27/05/2015 22:31","01/06/2015 11:09","04/06/2015 12:57","10/06/2015 04:00","15/06/2015 03:23","19/06/2015 05:37","23/06/2015 13:41","27/06/2015 15:43"],
                            'gender': ["male","female","female","male","male","female","female",np.nan,"male","male","female","male","female","female","male","female","male","female",np.nan,"male"],
                            'age': ["young","old","old","old","old","old",np.nan,"old","old","young","young","old","young","young","old",np.nan,"old","young",np.nan,np.nan]})

# Groupby based on 2 categorical variables
tempGroupby = tempDF.groupby(['gender','age'])

# Aggregate for each variable in each group using function defined above
dfAgg = tempGroupby.agg(lambda x: tempFuncAgg(x))

# Transform in list
dfAgg.applymap(lambda x: list(x))

print(dfAgg)

使用Pandas grouopby agg函数将集合转换为列表会导致'ValueError：函数不会减少'

1 个答案: