Question

我有这段代码：

model_nn <- train(
  Y ~ ., training,
  method = "nnet",
  metric = "ROC",
  trControl = trainControl(
    method = "cv", 
    number = 10,
    verboseIter = TRUE,
    classProbs = TRUE,
    summaryFunction = twoClassSummary
  )
)

nnprediction <- predict(model_nn, testing)
cmnn <-confusionMatrix(nnprediction,testing$Y)
print(cmnn)

哪个有效。但是，我无法使用confusionMatrix命令评估ROC指标性能的好坏程度。我如何评估它，以便尝试一组不同的变量和/或机器学习算法来提高ROC性能？

PS：因变量是两个类的因子。

Answer 1

只需输入mysqli_select_db("wp_newaddress", $con); if(isset($_POST['submit'])) { $id = $_POST['$userid']; $product = $_POST['sku']; $address = $_POST['address']; $sql = "UPDATE wp_newaddress SET address='address' WHERE user='$userid' AND product='sku'"; echo "<p><h5>Change address:</h5>"; $loop = new WP_Query( $args ); echo '<br><select name="sku">'; echo '<option>-- Select product--</option>'; while ( $loop->have_posts() ) : $loop->the_post(); global $product; echo '<option value=' . $product->get_sku() . '>' . $product->get_sku() . ' </option>'; endwhile; echo '</select>'; echo '<br><input type="text" value="Insert new address here" id="address" size="40" />'; echo '<br><button type="submit" name="submit">Change address</button>'; $retval = mysqli_query( $sql, $con ); if(! $retval ) { die('Could not update data: ' . mysqli_error()); } echo "Updated address successfully\n";即可获得培训期间使用的不同设置的AUC分数;这是一个例子，使用model_nn数据的前100个记录（2个类）：

iris

结果：

library(caret)
library(nnet)

data(iris)
iris_reduced <- iris[1:100,]
iris_reduced <- droplevels(iris_reduced, "virginica")

model_nn <- train(
  Species ~ ., iris_reduced,
  method = "nnet",
  metric = "ROC",
  trControl = trainControl(
    method = "cv", 
    number = 5,
    verboseIter = TRUE,
    classProbs = TRUE,
    summaryFunction = twoClassSummary
  )
)

model_nn

顺便说一句，这里的术语“ROC”有点误导：返回的当然不是ROC（它是曲线，而不是数字），而是ROC曲线下的面积，即AUC（在

Neural Network 

100 samples
  4 predictors
  2 classes: 'setosa', 'versicolor' 

No pre-processing
Resampling: Cross-Validated (5 fold) 
Summary of sample sizes: 80, 80, 80, 80, 80 
Resampling results across tuning parameters:

  size  decay  ROC  Sens  Spec
  1     0e+00  1.0  1.0   1   
  1     1e-04  0.8  0.8   1   
  1     1e-01  1.0  1.0   1   
  3     0e+00  1.0  1.0   1   
  3     1e-04  1.0  1.0   1   
  3     1e-01  1.0  1.0   1   
  5     0e+00  1.0  1.0   1   
  5     1e-04  1.0  1.0   1   
  5     1e-01  1.0  1.0   1   

ROC was used to select the optimal model using  the largest value.
The final values used for the model were size = 1 and decay = 0.1.

中使用metric='AUC'具有相同的效果）。

评估ROC指标，插入符号包 - R.

1 个答案: