我有一个以1分钟为增量的时间序列数据。我已经编写了代码,但是由于我拥有大量数据(超过1M行),因此遍历每行的时间太长。数据如下所示:
ReferenceEditorSearchFacade
它看起来像什么:
t0 = as.POSIXlt("2018-12-23 00:01:00")
t0 = t0+seq(60,60*10,60)
p1 = seq(5,5*10,5)
p2 = seq(7,7*10,7)
m0 = cbind(p1,p2)
rownames(m0) = as.character(t0)
我想通过在每分钟之前添加11行(55秒)并将该值从最新值开始沿转,以5秒为增量将数据转换为5秒。就像这样:
> head(m0)
p1 p2
2018-12-23 00:02:00 5 7
2018-12-23 00:03:00 10 14
2018-12-23 00:04:00 15 21
2018-12-23 00:05:00 20 28
2018-12-23 00:06:00 25 35
2018-12-23 00:07:00 30 42
我希望找到某种方法,而不必使用循环,而不必使用我不太熟悉的xts和/或data.table中的有效代码。
我尝试使用基数R中的> new0
p1 p2
2018-12-23 00:01:05 5 7
2018-12-23 00:01:10 5 7
2018-12-23 00:01:15 5 7
2018-12-23 00:01:20 5 7
2018-12-23 00:01:25 5 7
2018-12-23 00:01:30 5 7
2018-12-23 00:01:35 5 7
2018-12-23 00:01:40 5 7
2018-12-23 00:01:45 5 7
2018-12-23 00:01:50 5 7
2018-12-23 00:01:55 5 7
2018-12-23 00:02:00 5 7
2018-12-23 00:02:05 10 14
2018-12-23 00:02:10 10 14
2018-12-23 00:02:15 10 14
2018-12-23 00:02:20 10 14
2018-12-23 00:02:25 10 14
2018-12-23 00:02:30 10 14
2018-12-23 00:02:35 10 14
2018-12-23 00:02:40 10 14
2018-12-23 00:02:45 10 14
2018-12-23 00:02:50 10 14
2018-12-23 00:02:55 10 14
2018-12-23 00:03:00 10 14
函数,但是它不够快。
答案 0 :(得分:9)
由于您用data.table
标记了此内容:
library(data.table)
dt = as.data.table(m0, keep = T)[, rn := as.POSIXct(rn)]
dt[.(rep(rn, each = 12) - seq(0, 55, 5)), on = 'rn', roll = -Inf][order(rn)]
# rn p1 p2
# 1: 2018-12-23 00:01:05 5 7
# 2: 2018-12-23 00:01:10 5 7
# 3: 2018-12-23 00:01:15 5 7
# 4: 2018-12-23 00:01:20 5 7
# 5: 2018-12-23 00:01:25 5 7
# ---
#116: 2018-12-23 00:10:40 50 70
#117: 2018-12-23 00:10:45 50 70
#118: 2018-12-23 00:10:50 50 70
#119: 2018-12-23 00:10:55 50 70
#120: 2018-12-23 00:11:00 50 70
答案 1 :(得分:5)
这是在基数R中执行此操作的一种方法。首先,将您的数据转换为带有用于时间戳的显式列的数据框:
m0 <- as.data.frame(m0)
m0$t <- t0
p1 p2 t
1 5 7 2018-12-23 00:02:00
2 10 14 2018-12-23 00:03:00
3 15 21 2018-12-23 00:04:00
4 20 28 2018-12-23 00:05:00
5 25 35 2018-12-23 00:06:00
6 30 42 2018-12-23 00:07:00
7 35 49 2018-12-23 00:08:00
8 40 56 2018-12-23 00:09:00
9 45 63 2018-12-23 00:10:00
10 50 70 2018-12-23 00:11:00
然后merge
将此数据帧与1列时差(0到55)的数据帧:
m1 <- merge(m0, data.frame(diff = seq(0, 55, 5)))
最后,从时间戳列中减去差分列以创建新值:
m1$t2 <- with(m1, t - diff)
> m1[c(1, 20, 40), ]
p1 p2 t diff t2
1 5 7 2018-12-23 00:02:00 0 2018-12-23 00:02:00
20 50 70 2018-12-23 00:11:00 5 2018-12-23 00:10:55
40 50 70 2018-12-23 00:11:00 15 2018-12-23 00:10:45
答案 2 :(得分:4)
lubridate,padr
和tidyr的组合将带您到那里。我使用lubridate
来格式化日期,因此它可以和padr
一起使用。 padr
将缺少的日期时间值添加到数据框中。最后,使用提迪尔的fill
函数填充空白值。请注意,默认情况下,padr
的存储保护行数为100万行,但是您可以将此值设置得更高。
library(lubridate)
library(padr)
library(tidyr)
df1 <- data.frame(ymd_hms(t0), p1, p2)
df1 <- pad(df1, interval = "5 secs", start_val = lubridate::ymd_hms("2018-12-23 00:01:05"))
df1 <- fill(df1, p1, p2, .direction = "up")
head(df1, 15)
t0 p1 p2
1 2018-12-23 00:01:05 5 7
2 2018-12-23 00:01:10 5 7
3 2018-12-23 00:01:15 5 7
4 2018-12-23 00:01:20 5 7
5 2018-12-23 00:01:25 5 7
6 2018-12-23 00:01:30 5 7
7 2018-12-23 00:01:35 5 7
8 2018-12-23 00:01:40 5 7
9 2018-12-23 00:01:45 5 7
10 2018-12-23 00:01:50 5 7
11 2018-12-23 00:01:55 5 7
12 2018-12-23 00:02:00 5 7
13 2018-12-23 00:02:05 10 14
14 2018-12-23 00:02:10 10 14
15 2018-12-23 00:02:15 10 14
答案 3 :(得分:2)
基本方式:
m0 <- as.data.frame(m0)
time <- lapply(as.POSIXct(rownames(m0)), seq, by = "-5 sec", len = 12)
m1 <- cbind(TIME = Reduce(c, time), m0[rep(seq_len(nrow(m0)), each = 12), ])
row.names(m1) <- NULL
head(m1)
# TIME p1 p2
# 1 2018-12-23 00:02:00 5 7
# 2 2018-12-23 00:01:55 5 7
# 3 2018-12-23 00:01:50 5 7
# 4 2018-12-23 00:01:45 5 7
# 5 2018-12-23 00:01:40 5 7
# 6 2018-12-23 00:01:35 5 7
注意:输出中的变量TIME
取反。
答案 4 :(得分:2)
这是一种通用的xts解决方案,该解决方案应该适用于与问题中指定的参数不同的参数。
# convert m0 to xts
x0 <- as.xts(m0)
# create empty xts object with observations at all time points you want
nobs <- 11
nsec <- 5
y0 <- xts(, index(x0) - rep(seq_len(nobs) * nsec, each = nrow(x0)))
# merge data with desired index observations
new0 <- merge(x0, y0)
# carry the current value backward
new0 <- na.locf(new0, fromLast = TRUE)
head(new0, 20)
# p1 p2
# 2018-12-23 00:01:05 5 7
# 2018-12-23 00:01:10 5 7
# 2018-12-23 00:01:15 5 7
# 2018-12-23 00:01:20 5 7
# 2018-12-23 00:01:25 5 7
# 2018-12-23 00:01:30 5 7
# 2018-12-23 00:01:35 5 7
# 2018-12-23 00:01:40 5 7
# 2018-12-23 00:01:45 5 7
# 2018-12-23 00:01:50 5 7
# 2018-12-23 00:01:55 5 7
# 2018-12-23 00:02:00 5 7
# 2018-12-23 00:02:05 10 14
# 2018-12-23 00:02:10 10 14
# 2018-12-23 00:02:15 10 14
# 2018-12-23 00:02:20 10 14
# 2018-12-23 00:02:25 10 14
# 2018-12-23 00:02:30 10 14
# 2018-12-23 00:02:35 10 14
# 2018-12-23 00:02:40 10 14