熊猫 - 事件分离 - .iloc iteritem()?

时间:2014-08-21 08:32:33

标签: python pandas

我有一个带结构的sample_data.txt。

Precision= Waterdrops

2009-11-17 14:00:00,4.9,
2009-11-17 14:30:00,6.1,
2009-11-17 15:00:00,5.3,
2009-11-17 15:30:00,3.3,
2009-11-17 16:00:00,4.9,

我需要将数据与大于零的值分开,并确定timepam大于2小时的更改(事件)。到目前为止我写道:

file_path  = 'sample_data.txt'
df = pd.read_csv(file_path, skiprows = [num for (num,line) in enumerate(open(file_path),2) if 'Precision=' in line][0],
                 parse_dates =  True,index_col = 0,header= None, sep =',',
                 names = ['meteo', 'empty'])
df['date'] = df.index
df = df.drop(['empty'], axis=1)
df = df[df.meteo>20]
df['diff'] = df.date-df.date.shift(1)
df['sections'] = (diff > np.timedelta64(2, "h")).astype(int).cumsum()

从上面的代码我得到:

                   meteo    date                diff       sections
2009-12-15 12:00:00 23.8    2009-12-15 12:00:00 NaT         0
2009-12-15 13:00:00 23.0    2009-12-15 13:00:00 01:00:00    0

如果我使用:

df.date.iloc[[0, -1]].reset_index(drop=True)

我明白了:

0   2009-12-15 12:00:00
1   2012-12-05 16:00:00
Name: date, dtype: datetime64[ns]

我的example_data.txt的开始日期和结束日期。

如何为每个df ['部分']类别获取.iloc [[0,-1]] .reset_index(drop = True)?

我试过.apply:

def f(s):
    return s.iloc[[0, -1]].reset_index(drop=True)

df.groupby(df['sections']).apply(f)

我得到:IndexError:位置索引器超出范围

1 个答案:

答案 0 :(得分:1)

我不知道您使用drop_index()恶作剧的原因。从

开始,我的过程会更直接
df

   sections       meteo      date      diff
0         0  2009-12-15  12:00:00       NaT
1         0  2009-12-15  13:00:00  01:00:00
0         1  2009-12-15  12:00:00       NaT
1         1  2009-12-15  13:00:00  01:00:00

要做(确保sort('sections', 'date')确实iloc[0,-1]实际上是开始和结束,否则只需使用min()max()

def f(s):
    return s.iloc[[0, -1]]['date']
df.groupby('sections').apply(f)

date             0         1
sections                    
0         12:00:00  13:00:00
1         12:00:00  13:00:00

或者,作为一种更简化的方法

df.groupby('sections')['date'].agg([np.max, np.min])
              amax      amin
sections                    
0         13:00:00  12:00:00
1         13:00:00  12:00:00