Python数据框在具有相同条件的多个列上应用过滤器?

时间:2016-05-19 12:41:42

标签: python pandas filter group-by multiple-columns

这是我的熊猫数据框。

new_data =

    name   duration01        duration02          orz01      orz02     
    ABC   1 years 6 months    5 months           Nan        Google     
    XYZ   4 months            3 years 2 months   Google      Zensar      
    TYZ   4 months            4 years            Google In   Google   
    OPI   2 months            3 months           Nan       accenture    
    NRM   9 months            3 years            Google      Zensar     

我想找出在Google工作的员工的姓名,并且持续时间为几个月。这些值是否包含在多列中?如何在多列上应用过滤器?

duration01 => orz01(员工在orz01上花了多少个月/年) duration02 => orz02(员工在orz02上花了多少个月/年)

总共有10个orz和10个相应的持续时间列。

我试过下面的代码

# Selected the required columns
orz_cols = new_data.columns[new_data.columns.str.contains('orz')]

new_data [ new_data[orz_cols].apply(lambda x: x.str.contains('Google')) ]

但它没有打印正确的数据?

我如何实现这个

我想要输出如下

name  Total_duration_in Google_in_Months
ABC   5 months
XYZ   4 months
TYZ   52 months  

使用@Stefan给出的第一部分我在下面的部分将年份转换为数月

# filter the data

Google_Data = dt1[dt1['orz'].str.contains('Google')]

dur = []

for i in range(0,len(Google_Data['duration'])):
    dur.append(Google_Data['duration'][i].split())

months_list = []

for i in range(0,len(dur)):
    #print dur[i]
    if dur[i][1] == 'years':
        if len(dur[i]) > 2:
            val1 = int(dur[i][0]) * 12 + int(dur[i][2])
            val11 = str(val1)+" months"
            months_list.append(val11)
        else:
            val2 = int(dur[i][0]) * 12
            val22 = str(val2)+" months"
            months_list.append(val22)
    else:
        val3 = dur[i][0]+" months"
        months_list.append(val3)

months_list[:3]

# Concat
df2 = pd.DataFrame(months_list,index=Google_Data.index.copy())

Google_duration = pd.concat([Google_Data, df2], axis=1)


Output :

                    organization                      Duration_In_Months
name        
Aparna Arora        Google Headstrong Capital Markets   60 months
Aparna Dasgupta     Google                              24 months
Aparna Dhar         Google India Ltd                    56 months

现在我想执行最后一步,即通过对名称进行分组来获取总和,但这里的名称是'是索引。我很难得到这笔钱。

这是我正在尝试的

# Splitting the Duration_In_Months to get only number values
# Its returning the type as 'str'

Google_duration1 = Google_duration.Duration_In_Months.apply(lambda x : x.split()[0])

# apply groupby

Genpact_dur2.index.groupby(Genpact_dur2['Duration_In_Months'])

我如何组合索引并获得总和?

谢谢,

2 个答案:

答案 0 :(得分:0)

你可以这样做:

设置index并让columns合并:

df.set_index('name', inplace=True)    
orz_cols = [col for col in df.columns if col.startswith('orz')]
duration_cols = [col for col in df.columns if col.startswith('duration')]
merge_cols = zip(orz_cols, duration_cols)

使用pd.concat()重塑和重命名:

long_df = pd.concat([df.loc[:, cols].rename(columns={col: col[:-2] for col in orz_cols + duration_cols}) for cols in merge_cols])

消除非Google orz条目:

long_df = long_df[long_df.orz.str.contains('Google')]

根据duration和&而来计算month year long_df.duration = long_df.duration.str.split().apply(lambda x: int(x[0]) if x[1] == 'months' else int(x[0]) * 12)

name

long_df.groupby(level='name')['duration'].sum() duration name ABC 5 NRM 9 TYZ 52 XYZ 4 求和:

$return = [];
foreach ($data as $key => $value) {
    foreach ($value as $innerKey => $innerValue) {
        if (!isset($return[$innerKey])) {
            $return[$innerKey] = [];
        }
        $return[$innerKey][] = $innerValue;
    }
}

var_dump($return);
die();

答案 1 :(得分:0)

考虑使用pandas.melt进行重新整形,然后使用np.where()有条件地分析多年和数月的值。最后,由 Google 组织汇总。

import pandas as pd
import numpy as np

...
# LIST OF SUBSET COLUMNS
durationCols = [c for c in df.columns if 'duration' in c ]
orzCols = [c for c in df.columns if 'orz' in c ]

# MELT AND MERGE
df = pd.merge(pd.melt(df, id_vars=['name'], value_vars=durationCols,
                  var_name=None, value_name='duration'),
              pd.melt(df, id_vars=['name'], value_vars=orzCols,
                  var_name=None, value_name='orz'),
              right_index=True, left_index=True, on=['name'])[['name', 'duration', 'orz']]

# DURATION CONDITIONAL CALCULATION (YEAR + MONTH)
df['actual_dur'] = np.where(df['duration'].str.contains('year'),
                            df['duration'].str[:1], 0).astype(int) * 12 + \
                   np.where(df['duration'].str.contains('year.*month'),
                            df['duration'].str[8:9],
                            np.where(df['duration'].str.contains('month'),
                                     df['duration'].str[:1], 0)).astype(int)

df['orz'] = np.where(df['orz']\
                     .str.contains('Google'), 'Google', df['orz'])    

# SUM DURATION AND OUTPUT DF
df = df[df['orz']=='Google'].groupby(['name','orz']).sum().reset_index()    
df = df[['name','actual_dur']]
df.columns = ['name', 'Total_duration_in Google_in_Months']

输出

#   name  Total_duration_in Google_in_Months
# 0  ABC                                   5
# 1  NRM                                   9
# 2  TYZ                                  52
# 3  XYZ                                   4