Question

我试图通过运行嵌套循环来复制名为 df 的熊猫数据框。我只是通过每一列中的唯一值过滤数据，然后将结果附加到一个空的数据框中。 df 中的行数为627974。 df_new 中的行数为2275。这是数据的前10行

    cid     run_bal     last_tran_date  mark_seg    province    local_body  branch              acc_type       int_rate     
0   11276   495000.0    2013/1/4    Institutional   4   Municipality       KULESHWOR BRANCH     NORMAL SAVINGS  0.065   
1   11277   1030000.0   2013/1/2    Institutional   1   Municipality       CHABAHIL BRANCH      NORMAL SAVINGS  0.065   
2   11278   1225000.0   2013/1/20   Institutional   1   Municipality       TINKUNE BRANCH       NORMAL SAVINGS  0.065   
3   11278   50000.0     2013/1/8    Institutional   4   Municipality       Jorpati Branch       NORMAL SAVINGS  0.065   
4   11279   320000.0    2013/1/28   Institutional   3   Gaunpalika         TRIPURESHWAR BRANCH  NORMAL SAVINGS  0.065   
5   11281   100000.0    2013/1/14   Institutional   3   Municipality       KANTIPATH BRANCH     NORMAL SAVINGS  0.06    
6   11282   10000.0     2013/1/1    Institutional   5   Municipality       Gwarko               NORMAL SAVINGS  0.06    
7   11283   10000.0     2013/1/28   Institutional   1   Municipality       SWOYAMBHU BRANCH     CALL DEPOSIT    0.04    
8   11284   200000.0    2013/1/15   Institutional   3   Gaunpalika         Anamnagar Branch     NORMAL SAVINGS  0.065   
9   11284   545000.0    2013/1/21   Institutional   3   Municipality       Bouddha Branch       CALL DEPOSIT    0.04

请找到下面设置的代码：

def bin_cus(df):
df_new = pd.DataFrame(columns= col_names)
mk_unq = df.mark_seg.unique()
for mk in mk_unq:
    df_f = df.loc[df.mark_seg == mk]
    prc_unq = df_f.province.unique()
    for prc in prc_unq:
        df_f = df_f.loc[df_f.province == prc]
        lb_unq = df_f.local_body.unique()
        for lb in lb_unq:
            df_f = df_f.loc[df_f.local_body == lb]
            brn_unq = df_f.branch.unique()
            for brn in brn_unq:
                df_f = df_f.loc[df_f.branch == brn]
                acc_unq = df_f.acc_type.unique()
                for acc in acc_unq:
                    df_f = df_f.loc[df_f.acc_type == acc]

                    df_new = pd.concat([df_new, df_f], axis = 0)
return df_new

Answer 1

您正在每个循环内的同一数据框中进行更改。如果您如下所示重写代码，它将解决您的问题。

dep_cus_pandas_df = dep_cus_spark_df.toPandas()
dep_cus_pandas_df["cus_seg"] = ""
col_names = dep_cus_pandas_df.columns
df = dep_cus_pandas_df
df_new = pd.DataFrame(columns= col_names)
mk_unq = df.mark_seg.unique()
for mk in mk_unq:
    df_f_m = df.loc[df.mark_seg == mk]
    prc_unq = df_f_m.province.unique()
    for prc in prc_unq:
        df_f_p = df_f_m.loc[df_f_m.province == prc]
        lb_unq = df_f_p.local_body.unique()
        for lb in lb_unq:
            df_f_lb = df_f_p.loc[df_f_p.local_body == lb]
            brn_unq = df_f_lb.branch.unique()
            for brn in brn_unq:
                df_f_brn = df_f_lb.loc[df_f_lb.branch == brn]
                acc_unq = df_f_brn.acc_type.unique()
                for acc in acc_unq:
                    df_f_acc = df_f_brn.loc[df_f_brn.acc_type == acc]
                    df_f_acc.cus_seg = pd.cut(df_f_acc.run_bal.sort_values(), 4, labels= ["min_q1", "q1_q2", "q2_q3", "q3_max"],
                                               duplicates = "drop")
                    df_new = pd.concat([df_new, df_f_acc], axis = 0)

dep_cus_spark_df_new = sc_sql.createDataFrame(df_new, schema=["cid", "run_bal", "last_tran_date",
                                                               "mark_seg", "province","local_body",
                                                               "branch", "acc_type", "int_rate", "cus_seg"])

在熊猫数据框中运行嵌套循环时丢失行

1 个答案: