处理r

时间:2019-06-20 05:23:25

标签: r time-series

我正在处理时间序列数据,我需要有连续的时间戳,但是在捕获数据时几乎没有丢失数据时间戳记点,如下所示,

DF

ID Time_Stamp             A           B                 C
1  02/02/2018 07:45:00   123          567               434     
2   02/02/2018 07:45:01    
.....                  ...

5   02/02/2018 07:46:00   
6   02/02/2018 07:46:10    112          2323            2323

如上面的示例df所示,时间戳记一直持续到row 5,但是在10 seconds5th之间丢失了6th row的捕获数据。我的数据框是关于60000 rows的信息,因此手动识别丢失的值很乏味。因此,我一直在寻找使用R

处理缺失值的自动化程序。

我的结果数据框如下,

ID Time_Stamp                     A           B                 C
1  02/02/2018 07:45:00           123          567               434     
2   02/02/2018 07:45:01    
.....                  ...

5   02/02/2018 07:46:00         mean(A1:A5)
5.1  02/02/2018 07:46:01        mean(A1:A5)  mean(B1:B5)         mean(C1:C5)
5.2  02/02/2018 07:46:02        mean(A1:A5)  mean(B1:B5)         mean(C1:C5) 
5.3  02/02/2018 07:46:03        mean(A1:A5)  mean(B1:B5)         mean(C1:C5) 
5.4  02/02/2018 07:46:04        mean(A1:A5)  mean(B1:B5)         mean(C1:C5)
5.5  02/02/2018 07:46:05        mean(A1:A5)  mean(B1:B5)         mean(C1:C5)
5.6  02/02/2018 07:46:06        mean(A1:A5)  mean(B1:B5)         mean(C1:C5)
5.7  02/02/2018 07:46:07        mean(A1:A5)  mean(B1:B5)         mean(C1:C5)
5.8  02/02/2018 07:46:08        mean(A1:A5)  mean(B1:B5)         mean(C1:C5)
5.9  02/02/2018 07:46:09        mean(A1:A5)  mean(B1:B5)         mean(C1:C5)
6   02/02/2018 07:46:10         112         2323            2323
6.1 02/02/2018 07:46:11         mean(A1:A15) mean(B1:B15)       mean(C1:C15)

甚至可以是该时间间隔中前几行的平均值。

 6.1 02/02/2018 07:46:11         mean(A14:A17) mean(B14:B17)      mean(C14:C17)

即缺少时间值。

我已经完成以下代码来获取整列的均值。

library(dplyr)
library(tidyr)

df %>%
  complete(Time_Stamp = seq(min(Time_Stamp), max(Time_Stamp), by = "sec")) %>%
  mutate_at(vars(A:C), ~replace(., is.na(.), mean(., na.rm = TRUE))) %>%
  mutate(ID = row_number())

它给出列中所有行的所有均值的输出。

就像跟随this code一样,效果很好,但我需要进行此修改。怎么做 请帮助

2 个答案:

答案 0 :(得分:1)

这里是tidyverse和基本R方法的组合,以实现结果。我们首先创建一个新列,并为每列累积平均值。然后,我们complete缺少的观察值,并用其他列中的相应方法替换NA

library(tidyverse)
cols <- c("A", "B", "C")

df1 <- df %>%
        mutate_at(cols, list(mean = ~cummean(.))) %>%
        complete(Time_Stamp = seq(min(Time_Stamp), max(Time_Stamp), by = "sec")) %>%
        fill(ends_with("mean")) %>%
        mutate(ID = row_number())

mean_cols <- grep("_mean$", names(df1))
df1[cols] <- Map(function(x, y) ifelse(is.na(x), y, x), df1[cols], df1[mean_cols])

df1[names(df)]

#     ID Time_Stamp              A     B     C
#   <int> <dttm>              <dbl> <dbl> <dbl>
# 1     1 2018-02-02 07:45:00  123   567   434 
# 2     2 2018-02-02 07:45:01  234   100   110 
# 3     3 2018-02-02 07:45:02  234   100   110 
# 4     4 2018-02-02 07:45:03  197   256.  218 
# 5     5 2018-02-02 07:45:04  197   256.  218 
# 6     6 2018-02-02 07:45:05  197   256.  218 
# 7     7 2018-02-02 07:45:06  197   256.  218 
# 8     8 2018-02-02 07:45:07  197   256.  218 
# 9     9 2018-02-02 07:45:08  197   256.  218 
#10    10 2018-02-02 07:45:09  197   256.  218 
#11    11 2018-02-02 07:45:10  112  2323  2323 
#12    12 2018-02-02 07:45:11  176.  772.  744.
#13    13 2018-02-02 07:45:12  176.  772.  744.
#14    14 2018-02-02 07:45:13  176.  772.  744.
#15    15 2018-02-02 07:45:14  176.  772.  744.
#16    16 2018-02-02 07:45:15  100    23    12 

如果您需要每个NA值的均值,它将变得更加简单

df %>%
  complete(Time_Stamp = seq(min(Time_Stamp), max(Time_Stamp), by = "sec")) %>%
  mutate_at(cols, ~ifelse(is.na(.), cummean(na.omit(.)), .)) %>%
  mutate(ID = row_number())

数据

df <- structure(list(ID = c(1, 2, 3, 4, 5), Time_Stamp = structure(c(1517557500, 
1517557501, 1517557502, 1517557510, 1517557515), class = c("POSIXct", 
"POSIXt"), tzone = "UTC"), A = c(123, 234, 234, 112, 100), B = c(567, 
100, 100, 2323, 23), C = c(434, 110, 110, 2323, 12)), row.names = c(NA, 
-5L), class = "data.frame")

看起来像

df
#  ID          Time_Stamp   A    B    C
#1  1 2018-02-02 07:45:00 123  567  434
#2  2 2018-02-02 07:45:01 234  100  110
#3  3 2018-02-02 07:45:02 234  100  110
#4  4 2018-02-02 07:45:10 112 2323 2323
#5  5 2018-02-02 07:45:15 100   23   12

答案 1 :(得分:1)

为此专门制作了一个非常直观的程序包,称为“ padr”。 我认为您会发现它可以满足您的需求: cran padr vignette