使用pandas数据帧按时间顺序转换日期

时间:2018-06-02 20:18:05

标签: python pandas datetime dataframe pandas-groupby

我需要帮助比较不同行和不同列中的日期,并确保它们遵循时间顺序。

首先,我根据 ID 列对数据进行分组。接下来,每个日期值都应该在将来发生。

第一组[1111 + A]包含错误,因为日期不按时间顺序排列:

1/1/2016  >  2/20/2016  > **2/19/2016** >  4/25/2016  >  **4/1/2016** >  5/1/2016

当前结果

      id       start         end group
0   1111  01/01/2016  02/20/2016     A
1   1111  02/19/2016  04/25/2016     A
2   1111  04/01/2016  05/01/2016     A
3   2345  05/01/2016  05/28/2016     B
4   2345  05/29/2016  06/28/2016     B
5   1234  08/01/2016  09/16/2016     F
6   9882  01/01/2016  08/29/2016     D
7   9992  03/01/2016  03/15/2016     C
8   9992  03/16/2016  08/03/2016     C
9   9992  05/16/2016  09/16/2016     C
10  9992  09/17/2016  10/16/2016     C
11  9992  10/17/2016  12/13/2016     C

答案应该是:

1/1/2016  >  2/20/2016  > **2/21/2016** >  4/25/2016  >  **4/26/2016** >     5/1/2016

所需的输出

      id       start         end group
0   1111  01/01/2016  02/20/2016     A
1   1111  02/21/2016  04/25/2016     A
2   1111  04/26/2018  05/01/2016     A
3   2345  05/01/2016  05/28/2016     B
4   2345  05/29/2016  06/28/2016     B
5   1234  08/01/2016  09/16/2016     F
6   9882  01/01/2016  08/29/2016     C
7   9992  03/01/2016  03/15/2016     C
8   9992  03/16/2016  08/03/2016     C
9   9992  08/04/2016  09/16/2016     C
10  9992  09/17/2016  10/16/2016     C
11  9992  10/17/2016  12/13/2016     C

任何帮助将不胜感激。

2 个答案:

答案 0 :(得分:2)

一种方法是将逻辑应用于每个组,然后连接您的组。

# convert series to datetime
df['start'] = pd.to_datetime(df['start'])
df['end'] = pd.to_datetime(df['end'])

# iterate groups and add results to grps list
grps = []
for _, group in df.groupby(['id', 'group'], sort=False):
    end_shift = group['end'].shift()
    group.loc[group['start'] <= end_shift, 'start'] = end_shift + pd.DateOffset(1)
    grps.append(group)

# concatenate dataframes in grps to build a single dataframe
res = pd.concat(grps, ignore_index=True)

print(res)

      id      start        end group
0   1111 2016-01-01 2016-02-20     A
1   1111 2016-02-21 2016-04-25     A
2   1111 2016-04-26 2016-05-01     A
3   2345 2016-05-01 2016-05-28     B
4   2345 2016-05-29 2016-06-28     B
5   1234 2016-08-01 2016-09-16     F
6   9882 2016-01-01 2016-08-29     D
7   9992 2016-03-01 2016-03-15     C
8   9992 2016-03-16 2016-08-03     C
9   9992 2016-08-04 2016-09-16     C
10  9992 2016-09-17 2016-10-16     C
11  9992 2016-10-17 2016-12-13     C

答案 1 :(得分:1)

我相信这应该有效:

# First make sure your column are datetimes:
df['start'] = pd.to_datetime(df['start'])
df['end'] = pd.to_datetime(df['end'])

# Get your new start times:
new_times = (df.groupby(['id', 'group'])
               .apply(lambda x: (x.end + pd.Timedelta(days=1)).shift())
               .reset_index(['id', 'group'], drop=True))

# put back into original dataframe
df.loc[new_times.notnull(), 'start'] = new_times[new_times.notnull()]

>>> df
      id      start        end group
0   1111 2016-01-01 2016-02-20     A
1   1111 2016-02-21 2016-04-25     A
2   1111 2016-04-26 2016-05-01     A
3   2345 2016-05-01 2016-05-28     B
4   2345 2016-05-29 2016-06-28     B
5   1234 2016-08-01 2016-09-16     F
6   9882 2016-01-01 2016-08-29     D
7   9992 2016-03-01 2016-03-15     C
8   9992 2016-03-16 2016-08-03     C
9   9992 2016-08-04 2016-09-16     C
10  9992 2016-09-17 2016-10-16     C
11  9992 2016-10-17 2016-12-13     C

<强>解释

new_times看起来像这样:

>>> new_times
0           NaT
1    2016-02-21
2    2016-04-26
5           NaT
3           NaT
4    2016-05-29
6           NaT
7           NaT
8    2016-03-16
9    2016-08-04
10   2016-09-17
11   2016-10-17

然后,您可以使用df.loc[new_times.notnull(), 'start'] = new_times[new_times.notnull()]查找new_times不为空的位置(即它不是给定组中的第一行),并将new_times插入原始{ {1}}栏。