Question

您好我的名字是Abhi，我正在使用插入符号来构建基于gbm树的模型。但是，我想使用roc作为我的指标

而不是准确性

这是我到目前为止的代码

myTuneGrid <- expand.grid(n.trees = 500,interaction.depth = 11,shrinkage = 0.1)
fitControl <- trainControl(method = "repeatedcv", number = 7,repeats = 1, verboseIter = FALSE,returnResamp = "all",classProbs = TRUE)
myModel <- train(Cover_Type ~ .,data = modelData,method = "gbm",trControl = fitControl,tuneGrid = myTuneGrid,metric='roc')

然而，当我运行此代码时，我收到警告

Warning message:
In train.default(x, y, weights = w, ...) :
The metric "roc" was not in the result set. Accuracy will be used instead.

如何强制我的模型使用roc而不是精确度。我在这里做错了什么？

Answer 1

这是源代码的github项目的链接？ https://github.com/rseiter/PracticalMLProject/blob/master/multiClassSummary.R

Answer 2

如果您在twoClassSummary()中指定trainControl并且还使用metric="ROC"（而不是代码中的method="roc"），它将起作用：

df = iris
df$Species =factor(ifelse(df$Species=="versicolor","v","o"))

fitControl <- trainControl(method = "cv",returnResamp = "all",
classProbs = TRUE,summaryFunction = twoClassSummary)

myModel <- train(Species ~ .,data = df,method = "gbm",trControl = fitControl,metric='ROC')

Stochastic Gradient Boosting 

150 samples
  4 predictor
  2 classes: 'o', 'v' 

No pre-processing
Resampling: Cross-Validated (10 fold) 
Summary of sample sizes: 135, 135, 135, 135, 135, 135, ... 
Resampling results across tuning parameters:

  interaction.depth  n.trees  ROC    Sens  Spec
  1                   50      0.988  0.98  0.92
  1                  100      0.980  0.97  0.94
  1                  150      0.972  0.96  0.94
  2                   50      0.984  0.97  0.94
  2                  100      0.976  0.96  0.92
  2                  150      0.960  0.97  0.92
  3                   50      0.984  0.97  0.94
  3                  100      0.968  0.98  0.92
  3                  150      0.968  0.96  0.92

Tuning parameter 'shrinkage' was held constant at a value of 0.1

Tuning parameter 'n.minobsinnode' was held constant at a value of 10
ROC was used to select the optimal model using the largest value.
The final values used for the model were n.trees = 50, interaction.depth =
 1, shrinkage = 0.1 and n.minobsinnode = 10.

R - Caret - 在模型训练中使用ROC代替精确度

2 个答案: