将NA值替换为时间系列中的相邻值或同一列 - data.table方法

时间:2015-11-06 08:06:06

标签: r data.table interpolation na missing-data

示例数据

df <- data.frame(id=c("A","A","A","A","B","B","B","B"),year=c(2014,2014,2015,2015),month=c(1,2),marketcap=c(4,6,2,6,23,2,5,34),return=c(NA,0.23,0.2,0.1,0.4,0.9,NA,0.6))

df1
   id year month marketcap return
1:  A 2014     1         4     NA
2:  A 2014     2         6   0.23
3:  A 2015     1         2   0.20
4:  A 2015     2         6   0.10
5:  B 2014     1        23   0.40
6:  B 2014     2         2   0.90
7:  B 2015     1         5     NA
8:  B 2015     2        34   0.60

所需数据

desired_df <- data.frame(id=c("A","A","A","A","B","B","B","B"),year=c(2014,2014,2015,2015),month=c(1,2),marketcap=c(4,6,2,6,23,2,5,34),return=c(0.23,0.23,0.2,0.1,0.4,0.9,0.75,0.6))

desired_df
  id year month marketcap return
1  A 2014     1         4   0.23
2  A 2014     2         6   0.23
3  A 2015     1         2   0.20
4  A 2015     2         6   0.10
5  B 2014     1        23   0.40
6  B 2014     2         2   0.90
7  B 2015     1         5   0.75
8  B 2015     2        34   0.60

我想通过id替换NA值与时间序列中的相邻值来插值返回。假设只有两个月:一年1,2。 (B,2015,1)的第二个NA被0.75 =(0.9 + 0.6)/ 2取代 由于没有以前的数据,因此(A,2014,1)的第一个NA被替换为0.23。

如果可能的话,

data.table解决方案是首选的

更新: 使用如下代码结构(适用于样本)

df[,returnInterpolate:=na.approx(return,rule=2), by=id]

我遇到了错误: 约(x [!na],y [!na],xout,...)中的错误:   需要至少两个非NA值才能进行插值

我想可能有一些id没有非NA值要插值。 。有什么建议?

2 个答案:

答案 0 :(得分:5)

library(data.table)
df <- data.frame(id=c("A","A","A","A","B","B","B","B"),
                 year=c(2014,2014,2015,2015),
                 month=c(1,2),
                 marketcap=c(4,6,2,6,23,2,5,34),
                 return=c(NA,0.23,0.2,0.1,0.4,0.9,NA,0.6))
setDT(df)
library(zoo)
df[, returnInterpol := na.approx(return, rule = 2), by = id]
#   id year month marketcap return returnInterpol
#1:  A 2014     1         4     NA           0.23
#2:  A 2014     2         6   0.23           0.23
#3:  A 2015     1         2   0.20           0.20
#4:  A 2015     2         6   0.10           0.10
#5:  B 2014     1        23   0.40           0.40
#6:  B 2014     2         2   0.90           0.90
#7:  B 2015     1         5     NA           0.75
#8:  B 2015     2        34   0.60           0.60

修改

如果您的群组只有NA个值,或者只有一个非NA,那么您可以这样做:

df <- data.frame(id=c("A","A","A","A","B","B","B","B","C","C","C","C"),
                 year=c(2014,2014,2015,2015),
                 month=c(1,2),
                 marketcap=c(4,6,2,6,23,2,5,34, 1:4),
                 return=c(NA,0.23,0.2,0.1,0.4,0.9,NA,0.6,NA,NA,0.3,NA))
setDT(df)
df[, returnInterpol := switch(as.character(sum(!is.na(return))),
                              "0" = return,
                              "1" = {na.omit(return)},  
                              na.approx(return, rule = 2)), by = id]

#     id year month marketcap return returnInterpol
#  1:  A 2014     1         4     NA           0.23
#  2:  A 2014     2         6   0.23           0.23
#  3:  A 2015     1         2   0.20           0.20
#  4:  A 2015     2         6   0.10           0.10
#  5:  B 2014     1        23   0.40           0.40
#  6:  B 2014     2         2   0.90           0.90
#  7:  B 2015     1         5     NA           0.75
#  8:  B 2015     2        34   0.60           0.60
#  9:  C 2014     1         1     NA           0.30
# 10:  C 2014     2         2     NA           0.30
# 11:  C 2015     1         3   0.30           0.30
# 12:  C 2015     2         4     NA           0.30

答案 1 :(得分:0)

简单的imputeTS解决方案,无需关心ID,将是:

library("imputeTS")
na.interpolate(df)

由于插补应该根据ID进行,因此稍微复杂一点-因为按ID过滤时,似乎常常剩下的值不足。我会采用罗兰(Roland)发布的解决方案,并在可能的情况下使用imputeTS::na.interpolation(),在其他情况下,也许可以使用imputeTS::na.mean()的总体均值或对总体边界imputeTS::na.random()的随机猜测。 >

在这种情况下,超越单变量时间序列插值/插值法可能也是一个好主意。还有许多其他变量可以帮助估计缺失值(如果存在相关性)。像AMELIA这样的软件包可以在这里提供帮助。