Question

在下面的代码中，我尝试搜索xgboost的不同超参数。

param_test1 = {
 'max_depth':list(range(3,10,2)),
 'min_child_weight':list(range(1,6,2))
}
predictors = [x for x in train_data.columns if x not in ['target', 'id']]
gsearch1 = GridSearchCV(estimator=XGBClassifier(learning_rate =0.1, n_estimators=100, max_depth=5,
                                                min_child_weight=1, gamma=0, subsample=0.8, colsample_bytree=0.8,
                                                objective= 'binary:logistic', n_jobs=4, scale_pos_weight=1, seed=27, 
                                                kvargs={'tree_method':'gpu_hist'}),
                    param_grid=param_test1, scoring='roc_auc', n_jobs=4, iid=False, cv=5, verbose=2)
gsearch1.fit(train_data[predictors], train_data['target'])

即使我使用kvargs={tree_method':'gpu_hist'}，我也没有在实施中获得加速。根据{{1}}，GPU并没有太多参与计算：

nvidia-smi

我在Ubuntu中使用以下命令安装了支持GPU的xgboost：

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 375.66                 Driver Version: 375.66                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 1080    Off  | 0000:01:00.0      On |                  N/A |
|  0%   39C    P8    10W / 200W |    338MiB /  8112MiB |      1%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0       961    G   /usr/lib/xorg/Xorg                             210MiB |
|    0      1675    G   compiz                                         124MiB |
|    0      2359    G   /usr/lib/firefox/firefox                         2MiB |
+-----------------------------------------------------------------------------+

可能的原因是什么？

Answer 1

我想修改两件事。在Ubuntu中安装xgboost，

make -j4

至于Vivek的观点，我希望你们查看“树木”方法＆＃39;＆＃39; tree_method＆＃39;参数如下。

http://xgboost.readthedocs.io/en/latest/parameter.html

Answer 2

尝试添加单个参数：updater =＆＃39; grow_gpu＆＃39;

Answer 3

我知道有点晚了，但是，如果正确安装了cuda，以下代码应该可以工作：

没有GridSearch：

import xgboost

xgb = xgboost.XGBClassifier(n_estimators=200, tree_method='gpu_hist', predictor='gpu_predictor')
xgb.fit(X_train, y_train)

使用GridSearch：

params = {
        'max_depth': [3,4,5,6,7,8,10],
        'learning_rate':[0.001, 0.003, 0.01,0.03, 0.1,0.3],
        'n_estimators':[50,100,200,300,500,1000],
        .... whatever ....
}
xgb = xgboost.XGBClassifier(tree_method='gpu_hist', predictor='gpu_predictor')
tuner = GridSearchCV(xgb, params=params)
tuner.fit(X_train, y_train)

# OR you can pass them in params also.

使用支持GPU的XGBClassifier无法加速

3 个答案: