在pandas系列中填充NA值并停止

时间：2014-03-10 01:46:15

标签： python pandas

我正在分析时间序列，并根据某些条件，我可以选出事件的开始或结束行。在这一点上，我的系列看起来像这样（为了简洁我省略了一些重复的值）：

设置

import numpy as np
import pandas
from pandas import Timestamp

datadict = {'event': {
  Timestamp('2010-01-01 00:20:00', tz=None): 'event start',
  Timestamp('2010-01-01 00:30:00', tz=None): '--',
  Timestamp('2010-01-01 00:40:00', tz=None): '--',
  Timestamp('2010-01-01 00:50:00', tz=None): '--',
  Timestamp('2010-01-01 01:00:00', tz=None): '--',
  Timestamp('2010-01-01 01:10:00', tz=None): 'event end',
  Timestamp('2010-01-01 01:20:00', tz=None): '--',
  Timestamp('2010-01-01 02:20:00', tz=None): '--',
  Timestamp('2010-01-01 02:30:00', tz=None): 'event start',
  Timestamp('2010-01-01 02:40:00', tz=None): '--',
  Timestamp('2010-01-01 02:50:00', tz=None): '--',
  Timestamp('2010-01-01 03:00:00', tz=None): '--',
  Timestamp('2010-01-01 03:10:00', tz=None): '--',
  Timestamp('2010-01-01 03:20:00', tz=None): '--',
  Timestamp('2010-01-01 03:30:00', tz=None): 'event end',
}}
data = pandas.DataFrame.from_dict(datadict)

                           event
2010-01-01 00:20:00  event start
2010-01-01 00:30:00           --
2010-01-01 00:40:00           --
2010-01-01 00:50:00           --
2010-01-01 01:00:00           --
2010-01-01 01:10:00    event end
2010-01-01 01:20:00           --
2010-01-01 02:20:00           --
2010-01-01 02:30:00  event start
2010-01-01 02:40:00           --
2010-01-01 02:50:00           --
2010-01-01 03:00:00           --
2010-01-01 03:10:00           --
2010-01-01 03:20:00           --
2010-01-01 03:30:00    event end

以下是我想要实现的目标（理想情况下没有for循环）

                           event  event number
2010-01-01 00:20:00  event start  1
2010-01-01 00:30:00           --  1
2010-01-01 00:40:00           --  1
2010-01-01 00:50:00           --  1
2010-01-01 01:00:00           --  1
2010-01-01 01:10:00    event end  1
2010-01-01 01:20:00           --  NA
2010-01-01 02:20:00           --  NA
2010-01-01 02:30:00  event start  2
2010-01-01 02:40:00           --  2
2010-01-01 02:50:00           --  2
2010-01-01 03:00:00           --  2
2010-01-01 03:10:00           --  2
2010-01-01 03:20:00           --  2
2010-01-01 03:30:00    event end  2
2010-01-01 03:40:00           --  NA
2010-01-01 03:50:00           --  NA

这是我尝试过的

对我的数据质量有一些乐观的假设，我可以像这样得到事件编号：

table = data[data.event != '--'].reset_index()
table['event number'] = 1 + np.floor(table.index / 2)
table = table.set_index('index')

                           event  event number
index                                         
2010-01-01 00:20:00  event start             1
2010-01-01 01:10:00    event end             1
2010-01-01 02:30:00  event start             2
2010-01-01 03:30:00    event end             2

然后，我可以join到原始数据框，fillna和method='ffill' 

data2 = data.join(table[['event number']])
data2['filled'] = data2['event number'].fillna(method='ffill')

                           event  event number  filled
2010-01-01 00:20:00  event start             1       1
2010-01-01 00:30:00           --           NaN       1
2010-01-01 00:40:00           --           NaN       1
2010-01-01 00:50:00           --           NaN       1
2010-01-01 01:00:00           --           NaN       1
2010-01-01 01:10:00    event end             1       1
2010-01-01 01:20:00           --           NaN       1 # <- d'oh
2010-01-01 02:20:00           --           NaN       1 # <- d'oh 
2010-01-01 02:30:00  event start             2       2
2010-01-01 02:40:00           --           NaN       2
2010-01-01 02:50:00           --           NaN       2
2010-01-01 03:00:00           --           NaN       2
2010-01-01 03:10:00           --           NaN       2
2010-01-01 03:20:00           --           NaN       2
2010-01-01 03:30:00    event end             2       2

问题

如您所见，事件之间的时间（01:20至02:20）与事件＃1相关联。

问题

是否有没有循环跳过这些部分？

1 个答案:

答案 0 :(得分：5)

您只需查看event start的数量和event end的数量的累计总和即可实现此目的：

>>> data['event number'] = (data.event == 'event start').cumsum()
>>> data
                           event  event number
2010-01-01 00:20:00  event start             1
2010-01-01 00:30:00           --             1
2010-01-01 00:40:00           --             1
2010-01-01 00:50:00           --             1
2010-01-01 01:00:00           --             1
2010-01-01 01:10:00    event end             1
2010-01-01 01:20:00           --             1
2010-01-01 02:20:00           --             1
2010-01-01 02:30:00  event start             2
2010-01-01 02:40:00           --             2
2010-01-01 02:50:00           --             2
2010-01-01 03:00:00           --             2
2010-01-01 03:10:00           --             2
2010-01-01 03:20:00           --             2
2010-01-01 03:30:00    event end             2

现在你只需要在没有事件时设置为nan;但是那些地方对应的行是event start的累计总和等于event end的累计总和（换了1行）

>>> idx = data['event number'] == (data.event.shift(1) == 'event end').cumsum()
>>> data.loc[idx, 'event number'] = np.nan
>>> data
                           event  event number
2010-01-01 00:20:00  event start             1
2010-01-01 00:30:00           --             1
2010-01-01 00:40:00           --             1
2010-01-01 00:50:00           --             1
2010-01-01 01:00:00           --             1
2010-01-01 01:10:00    event end             1
2010-01-01 01:20:00           --           NaN
2010-01-01 02:20:00           --           NaN
2010-01-01 02:30:00  event start             2
2010-01-01 02:40:00           --             2
2010-01-01 02:50:00           --             2
2010-01-01 03:00:00           --             2
2010-01-01 03:10:00           --             2
2010-01-01 03:20:00           --             2
2010-01-01 03:30:00    event end             2

[15 rows x 2 columns]
相关问题
最新问题