在熊猫中按时间智能索引

时间:2018-05-15 08:18:35

标签: python pandas indexing

我有这样的数据框:

| Date    | Device | Status |
| 1990/01 | 50     | ON     |
| 1990/01 | 20     | ON     |
| 1990/03 | 25     | ON     |
| 1990/05 | 50     | OFF    |
| 2000/01 | 20     | OFF    |

基本上,我会在发生设备状态时记录设备状态的更改,但这不是定期的,所以'日期'专栏不是连续的#39; 我想用任何简单的逻辑来查询设备状态,例如

  1. df(device=50, date=1990/01) -> ON - easy
  2. df(device=50, date=1990/02) -> ON,设备50没有1990/02,但是当设备设置为特定状态时,除非没有变化,否则它将保留在其中
  3. df(device=50, date=1990/05) -> OFF
  4. df(device=50, date=2000/09) -> OFF
  5. df(device=50, date=1900/01) -> OFF,这是一个棘手的问题,在第一次开启之前,设备处于关闭状态
  6. 第二种情况具有不同的状态,但为了使问题简明扼要,我将保持不变。 除了第5点之外,行为完全相同。当我们假设首次登录状态之前的设备也处于ON状态时。 可能有更多的状态,但决定设备状态的逻辑完全相同。

    如何以pandastic方式做到这一点?

2 个答案:

答案 0 :(得分:1)

在我看来,必须在date_range中定义所选的所有可能日期,用于reindexpivot ed DataFrame的strftime。首先将NaN替换为NaN的前向填充和OFF的所有fillna值:

print (df)
      Date  Device Status
0  1990/01      50     ON
1  1990/01      20     ON
2  1990/03      25     ON
3  1990/05      50    OFF
4  1990/05      20    OFF <-changed for smaller output df

df['Date'] = pd.to_datetime(df['Date'])

rng = pd.date_range('1989-10-01', '1991-01-01', freq='MS')
df = df.pivot('Date','Device','Status').reindex(rng).ffill().fillna('OFF')
print (df)
Device       20   25   50
1989-10-01  OFF  OFF  OFF
1989-11-01  OFF  OFF  OFF
1989-12-01  OFF  OFF  OFF
1990-01-01   ON  OFF   ON
1990-02-01   ON  OFF   ON
1990-03-01   ON   ON   ON
1990-04-01   ON   ON   ON
1990-05-01  OFF   ON  OFF
1990-06-01  OFF   ON  OFF
1990-07-01  OFF   ON  OFF
1990-08-01  OFF   ON  OFF
1990-09-01  OFF   ON  OFF
1990-10-01  OFF   ON  OFF
1990-11-01  OFF   ON  OFF
1990-12-01  OFF   ON  OFF
1991-01-01  OFF   ON  OFF

如果需要原始格式的dates添加https://github.com/Smile-SA/elasticsuite/issues/912

,则最后
df.index = df.index.strftime('%Y/%m')
print (df)
Device    20   25   50
1989/10  OFF  OFF  OFF
1989/11  OFF  OFF  OFF
1989/12  OFF  OFF  OFF
1990/01   ON  OFF   ON
1990/02   ON  OFF   ON
1990/03   ON   ON   ON
1990/04   ON   ON   ON
1990/05  OFF   ON  OFF
1990/06  OFF   ON  OFF
1990/07  OFF   ON  OFF
1990/08  OFF   ON  OFF
1990/09  OFF   ON  OFF
1990/10  OFF   ON  OFF
1990/11  OFF   ON  OFF
1990/12  OFF   ON  OFF
1991/01  OFF   ON  OFF

编辑:

更一般的解决方案:

def get_status(df, device, check_date):
    check_date = pd.to_datetime(check_date)
    df['Date'] = pd.to_datetime(df['Date'])

    rng = pd.date_range(df['Date'].min(), df['Date'].max(), freq='MS')
    df = df.pivot('Date','Device','Status').reindex(rng).ffill().fillna('OFF')
    #print (df)

    if check_date < df.index.min():
        return 'OFF'
    elif check_date > df.index.max():
        return df.loc[df.index[-1], device]
    else:
        return df.loc[check_date, device]

print (get_status(df, 50, '1990/01'))
#ON
print (get_status(df, 50, '1990/02'))
#ON
print (get_status(df, 50, '1990/05'))
#OFF
print (get_status(df, 50, '1990/09'))
#OFF
print (get_status(df, 50, '1900/01'))
#OFF

答案 1 :(得分:1)

假设您的原始数据框名为status_df,并且您已将Date列转换为datetime.date类型。然后,您可以定义一个函数并使用status_df作为第一个参数和所需的devicecheck_date

来调用它
def get_status(df, device, check_date):
    # Filter only those entries for this device and with date <= check_date
    filtered_df = df[(df['Device']==device) & (df['Date']<=check_date)]
    # Now sort by date
    filtered_df.sort_values('Date', inplace=True)
    # Now the last entry of the filtered_df contains the active status at check_date
    # If it is empty choose a default values (say OFF)
    if len(filtered_df) < 1:
         return 'OFF'
    return filtered_df.iloc[-1]['Status']

<强>更新 如果您不想创建新的数据框,可以使用简单的try语句来完成

try:
    return df[(df['Device']==device) & (df['Date']<=check_date)].sort_values('Date').iloc[-1]['Status']
except IndexError:
    return 'OFF'