Question

我从数据库中按时间间隔收集数据。指标是计数器，因为它们不断增加。要获取给定时间的度量值，您必须从同一行的先前版本中减去一行。

示例：

                 TS INST_ID         EVENT WAIT_TIME_MILLI WAIT_COUNT
2014-01-29 17:20:36       1 log file sync               1     756873
2014-01-29 17:20:36       1 log file sync               2      15627
2014-01-29 17:20:36       1 log file sync               4       2925
2014-01-29 17:21:03       1 log file sync               1     761063
2014-01-29 17:21:03       1 log file sync               2      15659
2014-01-29 17:21:03       1 log file sync               4       2929

期望的输出：

                 TS INST_ID         EVENT WAIT_TIME_MILLI WAIT_COUNT
2014-01-29 17:21:03       1 log file sync               1       4190
2014-01-29 17:21:03       1 log file sync               2         32
2014-01-29 17:21:03       1 log file sync               4          4

TS是收集指标的时间。 INST_ID，EVENT和WAIT_TIME_MILLI是静态标识符。我想计算从一个TS到下一个TS的WAIT_COUNT的增量。

我已经简化了一些数据，但是如果重要的是有很多事件，可以是多个INST_ID。

这是测试数据框：

structure(list(TS = structure(c(1391034063.541, 1391034063.541, 
1391034063.541, 1391034036.136, 1391034036.136, 1391034036.136
), class = c("POSIXct", "POSIXt")), INST_ID = c(1, 1, 1, 1, 1, 
1), EVENT = c("log file sync", "log file sync", "log file sync", 
"log file sync", "log file sync", "log file sync"), WAIT_TIME_MILLI = c(1, 
2, 4, 1, 2, 4), WAIT_COUNT = c(761063, 15659, 2929, 756873, 15627, 
2925)), .Names = c("TS", "INST_ID", "EVENT", "WAIT_TIME_MILLI", 
"WAIT_COUNT"), class = "data.frame", row.names = c(NA, 6L))

Answer 1

如果您的数据是名为dat的data.frame

library(dplyr)
dat <- arrange(dat, WAIT_TIME_MILLI, TS)
dat <- group_by(dat, WAIT_TIME_MILLI)
dat <- mutate(dat, diff = WAIT_COUNT - lag(WAIT_COUNT))
filter(dat, !is.na(diff))

或：

library(dplyr)
dat %.%
  arrange(WAIT_TIME_MILLI, TS) %.%
  group_by(WAIT_TIME_MILLI) %.%
  mutate(diff = WAIT_COUNT - lag(WAIT_COUNT)) %.%
  filter(!is.na(diff))

Answer 2

@ mlt在data.table中实施的建议：

library(data.table)
dt <- data.table(df, key="TS")               # `key` orders dt by TS ascending
dt[, 
  list(
    TS=tail(TS, -1L),                        # all but first
    WAIT_COUNT=diff(WAIT_COUNT)),            # differences in WAIT_COUNT
  by=list(INST_ID, EVENT, WAIT_TIME_MILLI)   # split by these fields
]
#    INST_ID         EVENT WAIT_TIME_MILLI                  TS WAIT_COUNT
# 1:       1 log file sync               1 2014-01-29 17:21:03       4190
# 2:       1 log file sync               2 2014-01-29 17:21:03         32
# 3:       1 log file sync               4 2014-01-29 17:21:03          4

基本上，您通过INST_ID / EVENT / WAIT_TIME分解数据，然后为每个组diff分解所有值并删除第一个时间戳。

编辑：最后没注意到dput。

减去2个不同行中的数据

2 个答案: