Question

当我尝试预测在我的5＆amp;对于我们得到的10个经过培训的模型，我得到的ROC完全相同。我不确定我是否做错了，因为我无法找到使用CV来训练模型的测试或新数据的示例。

#### trainControl for both 5 and 10 fold CV
set.seed(2346)
fitControl_cv_5_roc <- trainControl(method= "cv", number = 5, 
                                    allowParallel = T, savePredictions = T, 
                                    classProbs = TRUE, 
                                    summaryFunction = twoClassSummary)
fitControl_cv_10_roc <- trainControl(method= "cv", number = 10, allowParallel = T, 
                                     savePredictions = T, classProbs = TRUE, 
                                     summaryFunction = twoClassSummary)

#### Building models for both 5 and 10 fold CV
set.seed(2346)
model.glm.upd.cv_5_roc <- train(FO ~ Length + Code + Type + Month + Priority + Job,
                                data = Merged_train, method = "glm", 
                                trControl= fitControl_cv_5_roc, metric ="ROC")

model.glm.upd.cv_10_roc <- train(FO ~ Length + Code + Type + Month + Priority + Job, 
                                 data = Merged_train, method = "glm", 
                                 trControl= fitControl_cv_10_roc, metric ="ROC")

#### Predictions
prediction.glm.up.cv_5_roc <- predict(model.glm.upd.cv_5_roc, newdata = Merged_test)

table(prediction.glm.up.cv_5_roc, Merged_test$FO)

prediction.glm.up.cv_5_roc  one zero
                      one  7172 1963
                      zero  801 1330

roc.curve(Merged_test$FO, prediction.glm.up.cv_5_roc, plotit = T) 
Area under the curve (AUC): 0.652

prediction.glm.up.cv_10_roc <- predict(model.glm.upd.cv_10_roc, 
                                       newdata = Merged_test)

table(prediction.glm.up.cv_10_roc,Merged_test$FO)

prediction.glm.up.cv_10_roc  one zero
                       one  7172 1963
                       zero  801 1330

roc.curve(Merged_test$FO, prediction.glm.up.cv_10_roc, plotit = T) 
Area under the curve (AUC): 0.652

从R中的插入符包中预测使用k-fold cv模型似乎不起作用

0 个答案: