我的所有模型都使用以下内容进行初始化:
def intiailize_clf_models(self):
model = RandomForestClassifier(random_state=42)
self.clf_models.append((model))
model = ExtraTreesClassifier(random_state=42)
self.clf_models.append((model))
model = MLPClassifier(random_state=42)
self.clf_models.append((model))
model = LogisticRegression(random_state=42)
self.clf_models.append((model))
model = xgb.XGBClassifier(random_state=42)
self.clf_models.append((model))
model = lgb.LGBMClassifier(random_state=42)
self.clf_models.append((model))
循环遍历模型并执行 k 折交叉验证:
def kfold_cross_validation(self):
clf_models = self.get_models()
models = []
self.results = {}
for model in clf_models:
self.current_model_name = model.__class__.__name__
cross_validate = cross_val_score(model, self.xtrain, self.ytrain, cv=4)
self.mean_cross_validation_score = cross_validate.mean()
print("Kfold cross validation for", self.current_model_name)
self.results[self.current_model_name] = self.mean_cross_validation_score
models.append(model)
每当我运行此交叉验证时,即使在不同模型上设置了随机状态后,我也会得到不同的结果。我想知道为什么我在交叉验证中得到不同的结果以及可以做些什么
答案 0 :(得分:3)
这是因为您没有为 k-fold 生成器设置 random_state。默认情况下,当您为 int
传递 cv
值时为
cross_validate = cross_val_score(model, self.xtrain, self.ytrain, cv=4)
cross_val_score
将使用不同的随机状态调用 (Stratified)KFold
,每次调用都会导致模型参数不同,从而导致不同的结果。
来自source file.的相关部分
cv: int, cross-validation generator or an iterable, default=None
Determines the cross-validation splitting strategy.
Possible inputs for cv are:
- None, to use the default 5-fold cross validation,
- int, to specify the number of folds in a `(Stratified)KFold`,
- :term:`CV splitter`,
- An iterable yielding (train, test) splits as arrays of indices.
For int/None inputs, if the estimator is a classifier and ``y`` is
either binary or multiclass, :class:`StratifiedKFold` is used. In all
other cases, :class:`KFold` is used.
为了解决这个问题,您可以传递您自己的交叉验证生成器,该生成器具有上述文档中所述的受控随机状态。例如:
# (code untested)
from sklearn.model_selection import StratifiedKFold
skf = StratifiedKFold(n_splits=4, random_state=42)
cross_validate = cross_val_score(model, self.xtrain, self.ytrain, cv=skf)
答案 1 :(得分:0)
我找到了问题的解决方案。
使用以下设置随机种子解决了问题:
seed = np.random.seed(22)