如何从DataFrame获取行块?

时间:2019-05-15 12:59:52

标签: python pandas

这是引用我的问题的数据框df

2018-03-04 21:25:19  8.0
2018-03-04 21:26:19  9.0
2018-03-04 21:27:19  9.5
2018-03-04 21:28:19  11.5
2018-03-04 21:29:19  11.9
2018-03-04 21:30:19  12.9
2018-03-04 21:31:19  14.2
2018-03-04 21:32:19  15.2
2018-03-04 21:33:19  15.5
2018-03-04 21:34:19  16.5
2018-03-04 21:35:19  14.8
2018-03-04 21:36:19  13.7
2018-03-04 21:37:19  11.0
2018-03-04 21:38:19  9.9

我有这段代码可以根据条件从pandas DataFrame检索行。条件是列col1的值应在10到15之间:

lower_bound = 10
upper_bound = 15

s_l=df["col1"].lt(lower_bound)
s_u=df["col1"].gt(upper_bound)

s = s_l | s_u

if (len(s)>0):
    df1=df[~s].copy()
    if df1.empty:
        print(None)
    else:
        s1=df1.groupby(s.cumsum()).date_time.transform(lambda x : x.max()-x.min()).dt.seconds
        print(df1.loc[(s1>1*60)])
else:
    print(None)

此函数应确定符合条件的两个行块:

2018-03-04 21:28:19  11.5
2018-03-04 21:29:19  11.9
2018-03-04 21:30:19  12.9
2018-03-04 21:31:19  14.2

2018-03-04 21:35:19  14.8
2018-03-04 21:36:19  13.7
2018-03-04 21:37:19  11.0

问题在于此代码将它们合并为一个块。我的最终目标是在第一个块2018-03-04 21:31:19中获得结束时间。 我该怎么办?

更新(基于Quang的答案):

df1 = df.copy()
s = df1[col].between(10,15)
if (len(s)>0):
    df1['block'] = (~s).cumsum()
    if df1.empty:
        print("None")
    else:
        new_df = df1[s].reset_index().set_index(['block', 'index'])
        s1 = new_df.groupby('block').date_time.transform(lambda x: x.max()-x.min()).dt.seconds
        print(new_df[s1>min_duration*60].columns) # date_time is among the columns!
        print(new_df[s1>min_duration*60].groupby('block').date_time.last())

错误:

  

KeyError:'date_time'

2 个答案:

答案 0 :(得分:2)

尝试:

s = df['col1'].between(10,15)
df['block'] = (~s).cumsum()
new_df = df[s].reset_index().set_index(['block', 'index'])

输出:

+-------+-------+---------------------+------+
|       |       |        date         | col1 |
+-------+-------+---------------------+------+
| block | index |                     |      |
+-------+-------+---------------------+------+
| 3     | 3     | 2018-03-04 21:28:19 | 11.5 |
|       | 4     | 2018-03-04 21:29:19 | 11.9 |
|       | 5     | 2018-03-04 21:30:19 | 12.9 |
|       | 6     | 2018-03-04 21:31:19 | 14.2 |
| 6     | 10    | 2018-03-04 21:35:19 | 14.8 |
|       | 11    | 2018-03-04 21:36:19 | 13.7 |
|       | 12    | 2018-03-04 21:37:19 | 11.0 |
+-------+-------+---------------------+------+

您可以通过以下方式选择跨度超过60秒的块:

s1 = new_df.groupby('block').date.transform(lambda x: x.max()-x.min()).dt.seconds
new_df[s1>60]

在我的代码中,date是时间戳列的名称。将其更改为您的实际数据。

答案 1 :(得分:0)

s = df['col1'].between(10,15)
split_dfs = []
for k,g in  df[s].groupby(df[s].index - np.arange(df[s].shape[0])):
    split_dfs.append(g)
last_value_in_first_block = split_dfs[0].loc[-1]