Question

由于数据大小我没能用PHP / MySQL或Excel解决我的问题，我现在正试着用R做我的第一步并且有点挣扎。问题是这样的：我有一个带有半年数据的倒数第二个CSV文件，如下所示：

metering,timestamp
123,2016-01-01 00:00:00
345,2016-01-01 00:00:01
243,2016-01-01 00:00:02
101,2016-01-01 00:00:04
134,2016-01-01 00:00:06

如您所见，每隔一段时间就会丢失几秒钟（不要问我，为什么在时间戳之前写入值，但这就是我收到数据的方式......）。现在我尝试计算缺少的值（=秒）。

所以我的想法是

创建一个正确的向量（包括所有逐秒时间戳），
将给定的CSV文件与该新矢量匹配，然后
总结所有时间戳，没有任何价值。

我设法使用以下代码进行第1步：

RegularTimeSeries <- seq(as.POSIXct("2016-01-01 00:00:00", tz = "UTC"), as.POSIXct("2016-01-01 00:00:30", tz = "UTC"), by = "1 sec")  
write.csv(RegularTimeSeries, file = "RegularTimeSeries.csv")

为了知道我做了什么，我还将矢量导出为CSV，如下所示：

"1",2016-01-01 00:00:00
"2",2016-01-01 00:00:01
"3",2016-01-01 00:00:02
"4",2016-01-01 00:00:03
"5",2016-01-01 00:00:04
"6",2016-01-01 00:00:05
"7",2016-01-01 00:00:06

不幸的是我不知道如何继续第2步和第3步。我发现了一些非常相似的例子（http://www.r-bloggers.com/fix-missing-dates-with-r/，R: Insert rows for missing dates/times），但作为一个总R菜鸟我努力翻译这些例子到我给出的每秒秒数据。

新手的一些提示非常有帮助 - 非常感谢你提前：）

Answer 1

在tidyverse中，

library(dplyr)
library(tidyr)

       # parse datetimes
df %>% mutate(timestamp = as.POSIXct(timestamp)) %>% 
    # complete sequence to full sequence from min to max by second
    complete(timestamp = seq.POSIXt(min(timestamp), max(timestamp), by = 'sec'))

## # A tibble: 7 x 2
##             timestamp metering
##                <time>    <int>
## 1 2016-01-01 00:00:00      123
## 2 2016-01-01 00:00:01      345
## 3 2016-01-01 00:00:02      243
## 4 2016-01-01 00:00:03       NA
## 5 2016-01-01 00:00:04      101
## 6 2016-01-01 00:00:05       NA
## 7 2016-01-01 00:00:06      134

如果您想要NA s的数量（即没有数据的秒数），请添加

%>% tally(is.na(metering))

## # A tibble: 1 x 1
##       n
##   <int>
## 1     2

Answer 2

您可以使用RegularTimeSeries和which检查您的时间序列中%in%的哪些值。首先从您的示例中创建BrokenTimeSeries：

RegularTimeSeries <- seq(as.POSIXct("2016-01-01 00:00:00", tz = "UTC"), as.POSIXct("2016-01-01 00:00:30", tz = "UTC"), by = "1 sec")
BrokenTimeSeries <- RegularTimeSeries[-c(3,6,9)] # remove some seconds

这将为您提供RegularTimeSeries中不在BrokenTimeSeries中的值的值：

> which(!(RegularTimeSeries %in% BrokenTimeSeries))
[1] 3 6 9

这将返回实际值：

> RegularTimeSeries[which(!(RegularTimeSeries %in% BrokenTimeSeries))]
[1] "2016-01-01 00:00:02 UTC" "2016-01-01 00:00:05 UTC" "2016-01-01 00:00:08 UTC"

也许我误解了你的问题，但是你可以计算丢失秒数，只需从length中减去你的破碎时间序列的RegularTimeSeries，或者得到上面两个结果向量中的任何一个的长度

> length(RegularTimeSeries) - length(BrokenTimeSeries)
[1] 3
> length(which(!(RegularTimeSeries %in% BrokenTimeSeries)))
[1] 3
> length(RegularTimeSeries[which(!(RegularTimeSeries %in% BrokenTimeSeries))])
[1] 3

如果要将文件合并在一起以查看缺失值，可以执行以下操作：

#data with regular time series and a "step"
df <- data.frame(
  RegularTimeSeries
)

df$BrokenTimeSeries[RegularTimeSeries %in% BrokenTimeSeries] <- df$RegularTimeSeries
df$BrokenTimeSeries <- as.POSIXct(df$BrokenTimeSeries, origin="2015-01-01", tz="UTC")

导致：

> df[1:12,]
     RegularTimeSeries    BrokenTimeSeries
1  2016-01-01 00:00:00 2016-01-01 00:00:00
2  2016-01-01 00:00:01 2016-01-01 00:00:01
3  2016-01-01 00:00:02                <NA>
4  2016-01-01 00:00:03 2016-01-01 00:00:02
5  2016-01-01 00:00:04 2016-01-01 00:00:03
6  2016-01-01 00:00:05                <NA>
7  2016-01-01 00:00:06 2016-01-01 00:00:04
8  2016-01-01 00:00:07 2016-01-01 00:00:05
9  2016-01-01 00:00:08                <NA>
10 2016-01-01 00:00:09 2016-01-01 00:00:06
11 2016-01-01 00:00:10 2016-01-01 00:00:07
12 2016-01-01 00:00:11 2016-01-01 00:00:08

Answer 3

如果您想要的只是丢失秒数，可以更简单地完成。首先查找时间范围内的秒数，然后减去数据集中的行数。这可以通过这些方式在R中完成：

n.seconds <- difftime("2016-06-01 00:00:00", "2016-01-01 00:00:00", units="secs")
n.rows <- nrow(my.data.frame)
n.missing.values <- n.seconds - n.rows

您可以更改数据框的时间范围和变量。

Answer 4

希望有所帮助

d <- (c("2016-01-01 00:00:01",
"2016-01-01 00:00:02",
"2016-01-01 00:00:03",
"2016-01-01 00:00:04",
"2016-01-01 00:00:05",
"2016-01-01 00:00:06",
"2016-01-01 00:00:10",
"2016-01-01 00:00:12",
"2016-01-01 00:00:14",
"2016-01-01 00:00:16",
"2016-01-01 00:00:18",
"2016-01-01 00:00:20",
"2016-01-01 00:00:22"))
d <- as.POSIXct(d)

for (i in 2:length(d)){
  if(difftime(d[i-1],d[i], units = "secs") < -1 ){
    c[i] <- d[i]
  }
  }
 class(c) <- c('POSIXt','POSIXct')
 c
 [1] NA                        NA                        NA                       
 NA                        NA                       
[6] NA                        "2016-01-01 00:00:10 EST" "2016-01-01 00:00:12    
EST" "2016-01-01 00:00:14 EST" "2016-01-01 00:00:16 EST"
[11] "2016-01-01 00:00:18 EST" "2016-01-01 00:00:20 EST" "2016-01-01    
00:00:22 EST"

R：在csv中找到缺少的时间戳

4 个答案: