如何在Pandas中使用groupby有效地添加新列?

时间:2016-02-17 04:43:43

标签: python pandas

我正在使用nba_py获取一些NBA比赛的记分牌数据。

以下是数据结构的示例:

    SEASON |     GAME_DATE_EST | GAME_SEQUENCE | GAME_ID | HOME_TEAM_ID | VISITOR_TEAM_ID | WINNER

0   2013    2013-10-05T00:00:00     1            11300001   12321         1610612760        V
1   2013    2013-10-05T00:00:00     2            11300002   1610612754    1610612741        V
2   2013    2013-10-05T00:00:00     3            11300003   1610612745    1610612740        V
3   2013    2013-10-05T00:00:00     4            11300004   1610612747    1610612744        H
4   2013    2013-10-06T00:00:00     1            11300005   12324         1610612755        V

您可以在此处找到部分数据:NBA Games Data

我的目标是创建并添加以下列的原始数据:

对于家庭团队:

   1. Total wins/losses for hometeam if hometeam plays at home ("HOMETEAM_HOME_WINS"/"HOMETEAM_HOME_LOSSES")
   2. Total wins/losses for hometeam if hometeam is visiting ("HOMETEAM_VISITOR_WINS"/"HOMETEAM_VISITOR_LOSSES")

访客_team:

   3. Total wins/losses for visitor_team if visitor_team plays at home ("VISITOR_TEAM_HOME_WINS"/"VISITOR_TEAM_HOME_LOSSES")
   4. Total wins/losses for visitor_team if visitor_team is visiting ("VISITOR_TEAM_VISITOR_WINS"/"VISITOR_TEAM_VISITOR_LOSSES")

我的第一个简单方法如下:

def get_home_team_home_wins(x):
    hometeam = x.HOME_TEAM_ID
    season = x.SEASON
    gid = x.name
    season_hometeam_games = grouped_seasons_hometeams.get_group((season, hometeam))
    home_games = season_hometeam_games[(season_hometeam_games.index < gid)]

    if not home_games.empty:
        try:
            home_wins = home_games.FTR.value_counts()["H"]
        except Exception as e:
            home_wins = 0
    else:
        home_wins = 0

grouped_seasons_hometeams = df.groupby(["SEASON", "HOME_TEAM_ID"])

df["HOMETEAM_HOME_WINS"] = df.apply(lambda x: get_home_team_home_wins(x), axis=1)

另一种方法是iterating over the rows

grouped_seasons = df.groupby("SEASON")
df["HOMETEAM_HOME_WINS"] = 0

current_season = 0
for index,row in df.iterrows():
    season = row.SEASON
    if season != current_season:
        current_season = season
        season_games = grouped_seasons.get_group(current_season)

    hometeam = row.HOME_TEAM_ID
    gid = row.name
    games = season_games[(season_games.index < gid)]
    home_games = games[(games.HOME_TEAM_ID == hometeam)]

    if not home_games.empty:
        try:
            home_wins = home_games.FTR.value_counts()["H"]
        except Exception as e:
            home_wins = 0
    else:
        home_wins = 0

    row["HOME_TEAM_HOME_WINS_4"] = home_wins
    df.ix[index] = row

更新1:

如果在家中玩游戏并且如果它在家中游戏,则有以下功能可以获得家庭的胜负。一个类似于visitor_team的人。

def get_home_team_home_wins_losses(x):
    hometeam = x.HOME_TEAM_ID
    season = x.SEASON
    gid = x.name

    games = df[(df.SEASON == season) & (df.index < gid)]
    home_team_home_games = games[(games.HOME_TEAM_ID == hometeam)]  


    # HOMETEAM plays at home
    if not home_team_home_games.empty:
        home_team_home_games_value_counts = home_team_home_games.FTR.value_counts()

        try:
            home_team_home_wins = home_team_home_games_value_counts["H"]
        except Exception as e:
            home_team_home_wins = 0

        try:
            home_team_home_losses = home_team_home_games_value_counts["V"]
        except Exception as e:
            home_team_home_losses = 0
    else:
        home_team_home_wins = 0
        home_team_home_losses = 0

    return [home_team_home_wins, home_team_home_losses]

def get_home_team_visitor_wins_losses(x):
    hometeam = x.HOME_TEAM_ID
    season = x.SEASON
    gid = x.name

    games = df[(df.SEASON == season) & (df.index < gid)]
    home_team_visitor_games = games[(games.VISITOR_TEAM_ID == hometeam)]

    # HOMETEAM visits
    if not home_team_visitor_games.empty:
        home_team_visitor_games_value_counts = home_team_visitor_games.FTR.value_counts()

        try:
            home_team_visitor_wins = home_team_visitor_games_value_counts["V"]
        except Exception as e:
            home_team_visitor_wins = 0

        try:
            home_team_visitor_losses = home_team_visitor_games_value_counts["H"]
        except Exception as e:
            home_team_visitor_losses = 0
    else:
        home_team_visitor_wins = 0
        home_team_visitor_losses = 0    

    return [home_team_visitor_wins, home_team_visitor_losses]

df["HOME_TEAM_HOME_WINS"], df["HOME_TEAM_HOME_LOSSES"] = zip(*df.apply(lambda x: get_home_team_home_wins_losses(x), axis=1))
df["HOME_TEAM_VISITOR_WINS"], df["HOME_TEAM_VISITOR_LOSSES"] = zip(*df.apply(lambda x: get_home_team_visitor_wins_losses(x), axis=1))
df["HOME_TEAM_WINS"] = df["HOME_TEAM_HOME_WINS"] + df["HOME_TEAM_VISITOR_WINS"]
df["HOME_TEAM_LOSSES"] = df["HOME_TEAM_HOME_LOSSES"] + df["HOME_TEAM_VISITOR_LOSSES"]

上述方法效率不高。所以,我正在考虑使用groupby,但它并不是很清楚。

每当我发现更高效的内容时,我都会添加更新。

有什么想法吗?感谢。

1 个答案:

答案 0 :(得分:0)

考虑使用transform()但首先有条件地创建HOMEWINNERVISITWINNER整数列。使用numpy.where()进行/ else计算时,注释掉更容易阅读,您可能/可能没有作为包使用。

请注意transform()会保留所有行,但会按ID进行汇总,因此特定HOME_TEAM_ID的每条记录都应重复这些汇总列中的值。

nbadf['VISITWINNER'] =  [1 if x == 'V' else 0 for x in nbadf['WINNER']]
#nbadf['VISITWINNER'] = np.where(nbadf['WINNER']=='V', 1, 0)

nbadf['HOMEWINNER'] = [1 if x == 'H' else 0 for x in nbadf['WINNER']]    
#nbadf['HOMEWINNER'] = np.where(nbadf['WINNER']=='H', 1, 0)

nbadf['HOME_TEAM_WINS'] = nbadf.groupby(['HOME_TEAM_ID','SEASON'])\ 
                                        ['HOMEWINNER'].transform(sum)
nbadf['HOME_TEAM_LOSSES'] = nbadf.groupby(['HOME_TEAM_ID','SEASON'])\
                                          ['VISITWINNER'].transform(sum)

nbadf['VISIT_TEAM_WINS'] = nbadf.groupby(['VISITOR_TEAM_ID','SEASON'])\
                                         ['VISITWINNER'].transform(sum)
nbadf['VISIT_TEAM_LOSSES'] = nbadf.groupby(['VISITOR_TEAM_ID','SEASON'])\
                                           ['HOMEWINNER'].transform(sum)

nbadf.drop(['HOMEWINNER', 'VISITWINNER'],inplace=True,axis=1)

#   SEASON  ...  WINNER  HOME_TEAM_WINS  HOME_TEAM_LOSSES  VISIT_TEAM_WINS  VISIT_TEAM_LOSSES
#0    2013  ...      V               0                 1                1                  0
#1    2013  ...      V               0                 1                1                  0
#2    2013  ...      V               0                 1                1                  0
#3    2013  ...      H               1                 0                0                  1
#4    2013  ...      V               0                 1                1                  0

现在,对于以后访问的主队的实例,反之亦然,考虑将ID与子集化数据帧合并(如果需要,更改列号)。这捕获了也是访客团队的主队。因此,在mergedf上运行上方的聚合(并使用此HOMEWINNERWINNER_x使用VISITWINNER计算相同的条件WINNER_y

# MERGES HOME SUBSET DF AND VISITOR SUBSET DF
mergedf = pd.merge(nbadf[[0,1,2,3,4,6]], nbadf[[0,1,2,3,5,6]],
                   left_on=['HOME_TEAM_ID'], right_on=['VISITOR_TEAM_ID'], how='inner')

mergedf['HOMETEAM_AS_VISITOR_WINS'] = mergedf.groupby(['VISITOR_TEAM_ID','SEASON_y'])\ 
                                                      ['VISITWINNER'].transform(sum)

mergedf['VISITORTEAM_AS_HOME_WINS'] = mergedf.groupby(['HOME_TEAM_ID','SEASON_x'])\ 
                                                      ['HOMEWINNER'].transform(sum)