Question

我正在尝试通过考虑时间戳记来创建新的df。具体来说，对于下面的df，我想返回上一行中integer中的Number为diff的所有行。然后，我想根据以下两个规则调整这些时间戳：

1) If integer increases, round to previous 15min mark
2) If integer decreases, keep the current timestamp

我不确定这是否是最有效的方法，但是我目前正在以这种方式进行。

df = pd.DataFrame({
    'Time' : ['1/1/1900 8:00:00','1/1/1900 9:59:00','1/1/1900 10:10:00','1/1/1900 12:21:00','1/1/1900 12:26:00','1/1/1900 13:00:00','1/1/1900 13:26:00','1/1/1900 13:29:00','1/1/1900 14:20:00','1/1/1900 18:10:00'],                 
    'Number' : [1,1,2,2,3,2,1,2,1,1],                      
    })

# First and last entry in df. This ensures the start/end of the subsequent
# df includes rows where the 'Number' increases/decreases.
first_time = df.loc[0,'Time']
last_time = df.loc[df.index[-1], 'Time']

# Insert 0 prior to first race
df.loc[-1] = [first_time, 0]  
df.index = df.index + 1  
df.sort_index(inplace=True) 

# Insert 0 after the last race
df.loc[len(df)] = last_time, 0

# Convert to datetime. Include new column that rounds all timestamps. If timestamp
# is within 10mins of nearest 15min, round to that point.
df['Time'] = pd.to_datetime(df['Time'])
df['New Time'] = df['Time'].sub(pd.Timedelta(11*60, 's')).dt.floor(freq='15T')

# Create separate df's. Inc contains all increased integers. Dec contains
# all decreases in integers  
df = df[df['Number'] != df['Number'].shift()]
Inc = df[df['Number'] > df['Number'].shift()]
Dec = df[df['Number'] < df['Number'].shift()]

del Inc['Time']
del Dec['New Time']
Inc.columns = ['Number','Time']

我的问题是何时在下一次减少之前将时间戳取整。很难理解，因此我将在问题下方添加一个图表以显示问题。

# Merge df's
df1 = pd.concat([Inc,Dec], sort = True)

# Sort so it's time ordered
df1['Time'] = pd.to_datetime(df1['Time'])
df1 = df1.iloc[pd.to_timedelta(df1['Time']).argsort()]

从本质上讲，如果在减小的整数的15分钟之内有一个增大的整数，则Number是不正确的。由于四舍五入，因此产生的时间戳记放错了位置。

当New Time在Number处增加到2时，图中的1:30:00没有记录。

我希望发生的是，如果整数的下降介于整数的15分钟之间，则忽略整数的减小。

x = df['Time']
x2 = df1['Time']

y = df['Number']
y2 = df1['Number']

plt.plot(x,y, drawstyle='steps-mid', label = 'Old Time')
plt.plot(x2,y2, drawstyle='steps-mid', label = 'New Time')
plt.legend()
plt.xticks(rotation = 45)

输出：

    Number                Time
1        1 1900-01-01 07:45:00
3        2 1900-01-01 09:45:00
5        3 1900-01-01 12:15:00
6        2 1900-01-01 13:00:00
8        2 1900-01-01 13:15:00 *Was previously 13:29:00
7        1 1900-01-01 13:26:00 *To be removed because within 15 of previous row
9        1 1900-01-01 14:20:00
11       0 1900-01-01 18:10:00

预期输出：

    Number                Time
1        1 1900-01-01 07:45:00
3        2 1900-01-01 09:45:00
5        3 1900-01-01 12:15:00
6        2 1900-01-01 13:00:00
8        2 1900-01-01 13:15:00
9        1 1900-01-01 14:20:00
11       0 1900-01-01 18:10:00

Answer 1

请尝试以下代码：

# if you want the last time in your dataframe to be zero, just execute the following line (as this is equivalent to adding a new column and deleting the old one):
df.iloc[-1, 1]= 0

# if you Time column is not of type datetime64, please execute the following line:
df['Time']= df['Time'].astype('datetime64')

# add some auxillary columns
df['row_id']= df.index                                         # this is needed for the delete indexer to avoid deleting adjusted rows that are joined with itself
df['increase']= df['Number'] > df['Number'].shift(1).fillna(0) # this is to identify the rows where the value increases and fillna(0) makes sure the value of the first row is regarded as an increase if it is larger than 0
df['Adjusted Time']= df['Time'].where(~df['increase'], df['Time'].sub(pd.Timedelta(11*60, 's')).dt.floor('15min')) # the Adjusted Time is the time we want to display later and also forms a range to delete (we want to delete other records later, if they lie between "Adjusted Time" and "Time"

# merge the ranges to identify the rows, we need to delete
get_delete_ranges= df[df['Time'] > df['Adjusted Time']]        # those are the ranges, for which we have to look if there is something else inbetween
df_with_del_ranges= pd.merge_asof(df, get_delete_ranges, left_on='Time', right_on='Adjusted Time', tolerance=pd.Timedelta('15m'), suffixes=['', '_del'])

# create an indexer for the rows to delete
del_row= (df_with_del_ranges['row_id_del'] != df_with_del_ranges['row_id']) & (df_with_del_ranges['Time'] >= df_with_del_ranges['Adjusted Time_del']) & (df_with_del_ranges['Time'] <= df_with_del_ranges['Time_del'])

# delete the rows in the overlapping ranges
df_with_del_ranges.drop(df_with_del_ranges[del_row].index, axis='index', inplace=True)
# remove the auxillary columns and restore the originals column names
df_with_del_ranges.drop([col for col in df_with_del_ranges if col not in ['Number', 'Adjusted Time']], axis='columns', inplace=True)
df_with_del_ranges.rename({'Adjusted Time': 'Time'}, axis='columns', inplace=True)

结果是：

In [131]: df_with_del_ranges
Out[131]: 
   Number                Time
0       1 1900-01-01 07:45:00
2       2 1900-01-01 09:45:00
4       3 1900-01-01 12:15:00
5       2 1900-01-01 13:00:00
7       2 1900-01-01 13:15:00
8       1 1900-01-01 14:20:00
9       0 1900-01-01 18:10:00

如果没有.loc[-1, 1]=0，则最后一行的Number列将包含1。

子集df可根据时间戳进行调整

1 个答案: