创建基于特定条件的新行并遍历熊猫列表

时间:2020-06-10 15:01:00

标签: python pandas

我有一个如下所示的df

B_ID   No_Show   Session  slot_num  Cumulative_no_show
    1     0.4       S1        1       0.4   
    2     0.3       S1        2       0.7      
    3     0.8       S1        3       1.5        
    4     0.3       S1        4       1.8       
    5     0.6       S1        5       2.4         
    6     0.8       S1        6       3.2       
    7     0.9       S1        7       4.1        
    8     0.4       S1        8       4.5   
    9     0.6       S1        9       5.1     
    12    0.9       S2        1       0.9    
    13    0.5       S2        2       1.4       
    14    0.3       S2        3       1.7        
    15    0.7       S2        4       2.4         
    20    0.7       S2        5       3.1          
    16    0.6       S2        6       3.7       
    17    0.8       S2        7       4.5        
    19    0.3       S2        8       4.8

在df上方创建的代码如下所示。

import pandas as pd
import numpy as np
df = pd.DataFrame({'B_ID': [1,2,3,4,5,6,7,8,9,12,13,14,15,20,16,17,19],
                   'No_Show': [0.4,0.3,0.8,0.3,0.6,0.8,0.9,0.4,0.6,0.9,0.5,0.3,0.7,0.7,0.6,0.8,0.3],
                   'Session': ['s1','s1','s1','s1','s1','s1','s1','s1','s1','s2','s2','s2','s2','s2','s2','s2','s2'],
                   'slot_num': [1,2,3,4,5,6,7,8,9,1,2,3,4,5,6,7,8],
                   })
df['Cumulative_no_show'] = df.groupby(['Session'])['No_Show'].cumsum()

和一个名为walkin_no_show = [0.3、0.4、0.3、0.4、0.3、0.4等等,长度为1000的列表]

使用upcumulative> 0.8从上方创建一个新行,紧随其后

 df[No_Show] = walkin_no_show[i]

及其会话和slot_num应该与前一个相同,并通过从前一个减去(1-walkin_no_show [i])创建一个名为u_cumulative的新列。

预期输出:

B_ID   No_Show   Session  slot_num  Cumulative_no_show    u_cumulative
    1     0.4       S1        1       0.4                 0.4
    2     0.3       S1        2       0.7                 0.7
    3     0.8       S1        3       1.5                 1.5
walkin1   0.3       S1        3       1.5                 0.8
    4     0.3       S1        4       1.8                 1.1      
walkin2   0.4       S1        4       1.8                 0.5
    5     0.6       S1        5       2.4                 1.1    
walkin3   0.3       S1        5       2.4                 0.4
    6     0.8       S1        6       3.2                 1.2      
walkin4   0.4       S1        6       3.2                 0.6
    7     0.9       S1        7       4.1                 1.5               
walkin5   0.3       S1        7       4.1                 0.8   
    8     0.4       S1        8       4.5                 1.2
walkin6   0.4       S1        8       4.5                 0.6
    9     0.6       S1        9       5.1                 1.2
    12    0.9       S2        1       0.9                 0.9
walkin1   0.3       S2        1       0.9                 0.2
    13    0.5       S2        2       1.4                 0.7           
    14    0.3       S2        3       1.7                 1.0
walkin2   0.4       S2        3       1.7                 0.4
    15    0.7       S2        4       2.4                 1.1
walkin3   0.3       S2        4       2.4                 0.4      
    20    0.7       S2        5       3.1                 1.1
walkin4   0.4       S2        5       3.1                 0.5       
    16    0.6       S2        6       3.7                 1.1
walkin5   0.3       S2        6       3.7                 0.4                    
    17    0.8       S2        7       4.5                 1.2
walkin6   0.4       S2        7       4.5                 0.6       
    19    0.3       S2        8       4.8                 0.9

我尝试了以下代码小的修改。正如@ Ben.T在下面回答的那样,提到了我的问题。

create new rows based the values of one of the column in pandas or numpy

感谢@ Ben.T。全部归功于您。.

def create_u_columns (ser):
    l_index = []
    arr_ns = ser.to_numpy()
    # array for latter insert
    arr_idx = np.zeros(len(ser), dtype=int)
    walkin_id = 1
    for i in range(len(arr_ns)-1):
        if arr_ns[i]>0.8:
            # remove 1 to u_no_show
            arr_ns[i+1:] -= (1-walkin_no_show[arr_idx])
            # increment later idx to add
            arr_idx[i] = walkin_id
            walkin_id +=1
    #return a dataframe with both columns
    return pd.DataFrame({'u_cumulative': arr_ns, 'mask_idx':arr_idx}, index=ser.index)

df[['u_cumulative', 'mask_idx']]= df.groupby(['Session']['Cumulative_no_show'].apply(create_u_columns)


# select the rows
df_toAdd = df.loc[df['mask_idx'].astype(bool), :].copy()
# replace the values as wanted
df_toAdd['No_Show'] = walkin_no_show[mask_idx]
df_toAdd['B_ID'] = 'walkin'+df_toAdd['mask_idx'].astype(str)
df_toAdd['u_cumulative'] -= 1
# add 0.5 to index for later sort
df_toAdd.index += 0.5 

new_df_0.8 = pd.concat([df,df_toAdd]).sort_index()\
           .reset_index(drop=True).drop('mask_idx', axis=1)

我也想遍历一个列表。我们可以在其中更改(arr_ns [i]> 0.8)[0.8,0.9,1.0]并创建3 df,例如new_df_0.8,new_df_0.9和new_df_1.0

2 个答案:

答案 0 :(得分:1)

您唯一需要考虑的技巧是增加索引值的方式。 这是一个解决方案:

walkin_no_show = [0.3,0.4,0.3,0.4,0.3]

TypeError: The added layer must be an instance of class Layer. Found: <keras.layers.core.Dense object at 0x7f736a163e10>

输出:

df = pd.DataFrame({'B_ID': [1,2,3,4,5],
                   'No_Show': [0.1,0.1,0.3,0.5,0.6],
                   'Session': ['s1','s1','s1','s2','s2'],
                   'slot_num': [1,2,3,1,2],
                   'Cumulative_no_show': [1.5, 0.4, 1.6, 0.3, 1.9]
                   })
df = df[['B_ID', 'No_Show', 'Session', 'slot_num', 'Cumulative_no_show']]
df['u_cumulative'] = df['Cumulative_no_show']

print(df.head())

然后:

   B_ID  No_Show Session  slot_num  Cumulative_no_show  u_cumulative
0     1      0.1      s1         1                 1.5           1.5
1     2      0.1      s1         2                 0.4           0.4
2     3      0.3      s1         3                 1.6           1.6
3     4      0.5      s2         1                 0.3           0.3
4     5      0.6      s2         2                 1.9           1.9

输出:

def Insert_row(row_number, df, row_value):
    # Starting value of upper half
    start_upper = 0

    # End value of upper half
    end_upper = row_number

    # Start value of lower half
    start_lower = row_number

    # End value of lower half
    end_lower = df.shape[0]

    # Create a list of upper_half index
    upper_half = [*range(start_upper, end_upper, 1)]

    # Create a list of lower_half index
    lower_half = [*range(start_lower, end_lower, 1)]

    # Increment the value of lower half by 1
    lower_half = [x.__add__(1) for x in lower_half]

    # Combine the two lists
    index_ = upper_half + lower_half

    # Update the index of the dataframe
    df.index = index_

    # Insert a row at the end
    df.loc[row_number] = row_value

    # Sort the index labels
    df = df.sort_index()

    # return the dataframe
    return df

walkin_count = 1
skip = False
last_Session = ''
i = 0
while True:
    row = df.loc[i]
    if row['Session'] != last_Session:
        walkin_count = 1
    last_Session = row['Session']

    values_to_append = ['walkin{}'.format(walkin_count), walkin_no_show[i],
                        row['Session'], row['slot_num'], row['Cumulative_no_show'], (1 - walkin_no_show[i])]

    if row['Cumulative_no_show'] > 0.8:
        df = Insert_row(i+1, df, values_to_append)
        walkin_no_show.insert(i+1, 0)
        walkin_count += 1
        i += 1
    i += 1
    if i == df.shape[0]:
        break
print(df)

我希望这会有所帮助。

Insert row at given position

导入的已使用函数

答案 1 :(得分:1)

IIUC,您可以通过以下方式实现:

def create_u_columns (ser, threshold_ns = 0.8):

    arr_ns = ser.to_numpy()
    # array for latter insert
    arr_idx = np.zeros(len(ser), dtype=int)
    walkin_id = 0 #start at 0 not 1 for list indexing
    for i in range(len(arr_ns)-1):
        if arr_ns[i]>threshold_ns:
            # remove 1 to u_no_show
            arr_ns[i+1:] -= (1-walkin_no_show[walkin_id]) #this is slightly different
            # increment later idx to add
            arr_idx[i] = walkin_id+1
            walkin_id +=1
    #return a dataframe with both columns
    return pd.DataFrame({'u_cumulative': arr_ns, 'mask_idx':arr_idx}, index=ser.index)

#create empty dict for storing the dataframes
d_dfs = {}
#iterate over the value for the threshold
for th_ns in [0.8, 0.9, 1.0]:
    #create a copy and do the same kind of operation
    df_ = df.copy()
    df_[['u_cumulative', 'mask_idx']]= \
        df_.groupby(['Session'])['Cumulative_no_show']\
           .apply(lambda x: create_u_columns(x, threshold_ns=th_ns))

    # select the rows
    df_toAdd = df_.loc[df_['mask_idx'].astype(bool), :].copy()
    # replace the values as wanted
    df_toAdd['No_Show'] = np.array(walkin_no_show)[df_toAdd.groupby('Session').cumcount()] 
    df_toAdd['B_ID'] = 'walkin'+df_toAdd['mask_idx'].astype(str)
    df_toAdd['u_cumulative'] -= (1 - df_toAdd['No_Show'])
    # add 0.5 to index for later sort
    df_toAdd.index += 0.5 

    d_dfs[th_ns] = pd.concat([df_,df_toAdd]).sort_index()\
                       .reset_index(drop=True).drop('mask_idx', axis=1)

然后,如果要访问数据框,则可以执行以下操作:

for th, df_ in d_dfs.items():
    print (th)
    print (df_.head(4))