MLR:如何在基准测试中提取特征选择

时间:2019-04-06 14:30:32

标签: r machine-learning cross-validation mlr

我想通过3倍交叉验证对mlr中的不同模型进行基准测试。在每一次折叠中,我都希望通过3折交叉验证再次进行操作,为每个模型选择一个特征,然后将最佳特征集传递给外部交叉验证。我注意到,MLR中的基准测试结果始终使用所有包含的功能。

如何从基准中提取每个折叠和每个模型中使用的功能,以及如何确保它们确实用于外部交叉验证折叠?

这是示例代码:

task_cv <- makeClassifTask(
  id = 'predict future outages',
  data = data, 
  target = 'targetVariable', 
  positive=1
)

vali_strat <- makeResampleDesc(method="CV",iters = 3)

featSelControl<- makeFeatSelControlSequential(same.resampling.instance = T,
                                                        method = "sbs",
                                                        tune.threshold = T,
                                                        alpha = 4,
                                                        beta = 4)

learner_nv <- makeLearner(
  id = 'Naive Bayes',
  cl = 'classif.naiveBayes'
)

learner_knn <- makeLearner(
  id = 'KNN',
  cl = 'classif.kknn'
)

featSel_nv <- makeFeatSelWrapper(learner = learner_nv,
                                          resampling = vali_strat,
                                          control = featSelControl,
                                          measures = acc

featSel_knn <- makeFeatSelWrapper(learner = learner_knn,
                                           resampling = vali_strat,
                                           control = featSelControl,
                                           measures = acc


learners <- list(featSel_nv,
                featSel_knn ) 

benchmark = benchmark(
  learners = learners,
  tasks = task_cv,
  resamplings = validation_strategy,
  measures = acc
)

benchmark$results$`predict future outages`$KNN.featsel$models[[1]]$features

我无法提取使用的功能,并且代码的最后一行指示始终使用所有功能,而不是通过featureSelection选择的功能。

0 个答案:

没有答案