让我们获取数据:
set.seed(42)
y <- rnorm(125)
x <- data.frame(runif(125), rexp(125))
我想对其进行 2 折交叉验证。所以:
library(caret)
model <- train(y ~ .,
data = cbind(y, x), method = "lm",
trControl = trainControl(method = "cv", number = 2)
)
model
Linear Regression
125 samples
2 predictor
No pre-processing
Resampling: Cross-Validated (2 fold)
Summary of sample sizes: 63, 62
Resampling results:
RMSE Rsquared MAE
1.091108 0.002550859 0.8472947
Tuning parameter 'intercept' was held constant at a value of TRUE
我想手动获取上面的这个 RMSE 值,以确保我完全理解交叉验证。
我目前的工作
正如我在上面看到的,我的样本被分成了两部分:62(1 折)和 63(第二折)。
#Training first model basing on first fold
model_1 <- lm(y[1:63] ~ ., data = x[1:63, ])
#Calculating RMSE for the first model
RMSE_1 <- RMSE(y[64:125], predict(model_1, newdata = x[64:125, ]))
#Training second model basing on second fold
model_2 <- lm(y[64:125] ~ ., data = x[64:125, ])
#Calculating RMSE for the second model
RMSE_2 <- RMSE(y[1:63], predict(model_1, newdata = x[1:63, ]))
mean(c(RMSE_1, RMSE_2))
1.023411
我的问题是 - 为什么我得到不同的 RMSE ?这个误差太大了,不能被视为估计误差——他们肯定是在以另一种方式计算它。你知道我在做什么不同吗?
答案 0 :(得分:1)
您使用的逻辑是正确的,但您需要进行两项更改:
RMSE_2
中的拼写错误model_2
这是更新的代码:
# the folds are kept in this part of the output (trial and error to find it haha)
model$control$index
f1 <- model$control$index[[1]]
f2 <- model$control$index[[2]]
# re-do your calculations but using the fold indexes, plus typo for RMSE_2
model_1 <- lm(y[f1] ~ ., data = x[f1, ])
#Calculating RMSE for the first model
RMSE_1 <- RMSE(y[f2], predict(model_1, newdata = x[f2, ]))
#Training second model basing on second fold
model_2 <- lm(y[f2] ~ ., data = x[f2, ])
#Calculating RMSE for the second model
RMSE_2 <- RMSE(y[f1], predict(model_2, newdata = x[f1, ]))
# matches now
mean(c(RMSE_1, RMSE_2))