如何排除NAs? (fitdist功能)

时间:2016-05-07 17:27:08

标签: r statistics na fitdistrplus

我有100x2数据帧DFN。在列DFN $ Lret上运行fitdist会出现错误消息"函数mle无法估计参数,错误代码为100"。我想是原因是最后一行包含一个NA。因此我运行fitdist除了NA,现在我得到错误"数据必须是长度大于1"的数字向量。有关如何解决此问题的任何想法?非常感谢。

DFN <- structure(list(LRet = c(0.0011, 0, -0.0026, 0, -0.0015, 0.0038, 3e-04, -0.0021, 4e-04, -0.001, 0, 0.0019, -6e-04, -8e-04, -5e-04, -8e-04, 3e-04, -5e-04, -0.0026, 0.0014, 7e-04, 0, -2e-04, 0.0011, -0.0025, 0.0042, 0.0022, -0.0017, -0.0058, 1e-04, 2e-04, 8e-04, -9e-04, -0.0014, -0.0014, -0.001, -0.0032, -0.0015, 6e-04, -8e-04, 0.001, -0.0014, -0.0017, -8e-04, -0.001, 0.0011, 0.0013, -0.001, 5e-04, 9e-04, -8e-04, -0.0025, 0.0027, 6e-04, 2e-04, -6e-04, 9e-04, -3e-04, -7e-04, 3e-04, 0, 2e-04, -6e-04, 1e-04, -1e-04, -7e-04, -8e-04, 7e-04, -1e-04, -7e-04, 7e-04, 8e-04, -8e-04, 8e-04, 0.0058, -1e-04, -5e-04, 0.0027, -0.0012, 7e-04, 7e-04, 0, 3e-04, -1e-04, 2e-04, -2e-04, -0.0013, -1e-04, 1e-04, -0.0011, 0.0013, 2e-04, -3e-04, -7e-04, 0, 0.0015, 1e-04, 3e-04, -0.0012, NA), LRetPct = c("0.11%", "0.00%", "-0.26%", "0.00%", "-0.15%", "0.38%", "0.03%", "-0.21%", "0.04%", "-0.10%", "0.00%", "0.19%", "-0.06%", "-0.08%", "-0.05%", "-0.08%", "0.03%", "-0.05%", "-0.26%", "0.14%", "0.07%", "0.00%", "-0.02%", "0.11%", "-0.25%", "0.42%", "0.22%", "-0.17%", "-0.58%", "0.01%", "0.02%", "0.08%", "-0.09%", "-0.14%", "-0.14%", "-0.10%", "-0.32%", "-0.15%", "0.06%", "-0.08%", "0.10%", "-0.14%", "-0.17%", "-0.08%", "-0.10%", "0.11%", "0.13%", "-0.10%", "0.05%", "0.09%", "-0.08%", "-0.25%", "0.27%", "0.06%", "0.02%", "-0.06%", "0.09%", "-0.03%", "-0.07%", "0.03%", "0.00%", "0.02%", "-0.06%", "0.01%", "-0.01%", "-0.07%", "-0.08%", "0.07%", "-0.01%", "-0.07%", "0.07%", "0.08%", "-0.08%", "0.08%", "0.58%", "-0.01%", "-0.05%", "0.27%", "-0.12%", "0.07%", "0.07%", "0.00%", "0.03%", "-0.01%", "0.02%", "-0.02%", "-0.13%", "-0.01%", "0.01%", "-0.11%", "0.13%", "0.02%", "-0.03%", "-0.07%", "0.00%", "0.15%", "0.01%", "0.03%", "-0.12%", " NA%")), .Names = c("LRet", "LRetPct"), class = "data.frame", row.names = 901:1000)

library(fitdistrplus)

#Following gives error code 100
f1 <- fitdist(DFN$LRet,"norm") 

#Following gives error code 100
f1 <- fitdist(DFN$LRet,"norm", na.rm=T)

#Following gives error data must be a numeric vector of length greater than 1"
f1 <- fitdist(na.exclude(DFN$LRet),"norm")
#Same result using na.omit

请注意,如果删除包含NA的最后一行,则上述代码可以正常工作。如果可以避免的话,我宁愿不必在运行fitdist之前消除最后一行。

编辑/更新:消除NA的最后一行确实首先解决了问题,但我现在无法一致地重现(即在消除最后一行后成功运行了几次代码,但并非总是如此) 。我想知道为什么。我尝试使用25x2数据帧,100x2和300x2,以及矢量,具有类似的结果。认为数据框或向量的大小可能是问题的一部分,因此不同大小的试验。

2 个答案:

答案 0 :(得分:6)

通过fitdist进行调试显示

 if (!(is.vector(data) & is.numeric(data) & length(data) > 1)) 
    stop("data must be a numeric vector of length greater than 1")

查看?is.vector

  如果'x'是指定的向量,则

'is.vector'返回'TRUE'        模式没有名称​​以外的

na.exclude及其亲属(na.omit等)将有关排除值的信息保存为属性,因此is.vector()变为FALSE ...

c()的一个副作用是删除名称以外的属性,因此is.vector(c(na.exclude(DFN$LRet)))TRUE,所以

fitdist(c(na.exclude(DFN$LRet)), "norm")

至少没有得到“必须是数字向量”错误 - 但我仍然得到“错误100”。进一步调查......

进一步挖掘fitdist的内容,似乎(正如@ 42-所示)optim()遇到了麻烦。具体来说,它实际上得到了答案,但当它试图计算解决方案的Hessian时,它会尝试标准差参数和barf的负值。

作为一个例子,这有效:

nn <- c(na.exclude(DFN$LRet))
fn <- function(x) -sum(dnorm(nn,mean=x[1],sd=x[2],log=TRUE))
optim(fn,par=c(mean(nn),sd(nn)),method="Nelder-Mead")

但这失败了:

optim(fn,par=c(mean(nn),sd(nn)),method="Nelder-Mead",hessian=TRUE)

答案 1 :(得分:4)

(还找到了代码写得不好的is.vector部分,但它没有解决错误。)fitdist函数似乎难以处理小方差的向量:

var( na.exclude(DFN$LRet))
[1] 2.220427e-06

你可以通过乘以10来解决这个问题:

> f1 <- fitdist(10*c(na.exclude(DFN$LRet)),"norm")
> f1
Fitting of the distribution ' norm ' by maximum likelihood 
Parameters:
          estimate  Std. Error
mean -0.0009090909 0.001490034
sd    0.0148256472 0.001032122

标准概率理论允许您更正这些估计值:平均值除以10,方差除以100(对于sd除以10)。经过更正的fitdist - 结果的估算值与样本值相当接近:

> all.equal( 0.0148256472/10 , sd(na.exclude(DFN$LRet) ) )
[1] "Mean relative difference: 0.005089095"