XGBoost培训不断失败

时间:2018-06-25 08:50:33

标签: apache-spark xgboost

我试图用Spark运行任何XGBoost代码,但没有成功。 我也尝试过粘贴粘贴的示例,但这些示例也不起作用。 假设我们有以下示例代码: https://docs.azuredatabricks.net/_static/notebooks/xgboost-simple-example.html

每当我运行XGBoost.trainWithDataFrame方法时,使用binary:logistic目标时,总是会出现此错误:

Check failed: base_score > 0.0f && base_score < 1.0f base_score must be in (0,1) for logistic loss

每次火车数据我总是会遇到这个错误,更改其他参数也无济于事。

使用不推荐使用的XGBoost.train时,我遇到了这些错误(我增加了回合数以生成更多此类错误):

[INFO] [06/25/2018 10:30:46.399] [RabitTracker-akka.actor.default-dispatcher-6] [akka://RabitTracker/user/Handler] [17] train-error:0.538462
[10:30:46] /xgboost/src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
[10:30:46] /xgboost/src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
[INFO] [06/25/2018 10:30:46.560] [RabitTracker-akka.actor.default-dispatcher-7] [akka://RabitTracker/user/Handler] [18] train-error:0.538462
[10:30:46] /xgboost/src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
[10:30:46] /xgboost/src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
[INFO] [06/25/2018 10:30:46.720] [RabitTracker-akka.actor.default-dispatcher-2] [akka://RabitTracker/user/Handler] [19] train-error:0.538462
[10:30:46] /xgboost/src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 0 extra nodes, 2[10:30:46] /xgboost/src/tree/updater_prune.cc: pruned nodes, max_depth=0
74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
[INFO] [06/25/2018 10:30:46.879] [RabitTracker-akka.actor.default-dispatcher-2] [akka://RabitTracker/user/Handler] [20] train-error:0.538462
[10:30:46] /xgboost/src/tree/updater_prune.cc:[10:30:46] /xgboost/src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
[INFO] [06/25/2018 10:30:47.040] [RabitTracker-akka.actor.default-dispatcher-4] [akka://RabitTracker/user/Handler] [21] train-error:0.538462
[10:30:47] /xgboost/src/tree/updater_prune.cc:[10:30:47] /xgboost/src/tree/updater_prune.cc:74: tree pruning end, 74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
[INFO] [06/25/2018 10:30:47.199] [RabitTracker-akka.actor.default-dispatcher-6] [akka://RabitTracker/user/Handler] [22] train-error:0.538462
[10:30:47] /xgboost/src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0[10:30:47] /xgboost/src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0

[INFO] [06/25/2018 10:30:47.359] [RabitTracker-akka.actor.default-dispatcher-6] [akka://RabitTracker/user/Handler] [23] train-error:0.538462
[10:30:47] /xgboost/src/tree/updater_prune.cc:[10:30:47] /xgboost/src/tree/updater_prune.cc:74: 74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
0
[INFO] [06/25/2018 10:30:47.519] [RabitTracker-akka.actor.default-dispatcher-10] [akka://RabitTracker/user/Handler] [24]    train-error:0.538462
[10:30:47] /xgboost/src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
[10:30:47] /xgboost/src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
[INFO] [06/25/2018 10:30:47.679] [RabitTracker-akka.actor.default-dispatcher-6] [akka://RabitTracker/user/Handler] [25] train-error:0.538462
[10:30:47] /xgboost/src/tree/updater_prune.cc:[10:30:47] /xgboost/src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
[INFO] [06/25/2018 10:30:47.839] [RabitTracker-akka.actor.default-dispatcher-2] [akka://RabitTracker/user/Handler] [26] train-error:0.538462
[10:30:47] /xgboost/src/tree/updater_prune.cc:[10:30:47] /xgboost/src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
[INFO] [06/25/2018 10:30:48.001] [RabitTracker-akka.actor.default-dispatcher-10] [akka://RabitTracker/user/Handler] [27]    train-error:0.538462
[10:30:48] /xgboost/src/tree/updater_prune.cc:[10:30:48] /xgboost/src/tree/updater_prune.cc74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0:74: tree pruning end, 1 roots, 
0 extra nodes, 2 pruned nodes, max_depth=0
[INFO] [06/25/2018 10:30:48.160] [RabitTracker-akka.actor.default-dispatcher-6] [akka://RabitTracker/user/Handler] [28] train-error:0.538462
[10:30:48] /xgboost/src/tree/updater_prune.cc:[10:30:4874: tree pruning end, 1 roots, ] /xgboost/src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
0 extra nodes, 2 pruned nodes, max_depth=0
[INFO] [06/25/2018 10:30:48.319] [RabitTracker-akka.actor.default-dispatcher-3] [akka://RabitTracker/user/Handler] [29] train-error:0.538462
[10:30:48] /xgboost/src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
[10:30:48] /xgboost/src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
[INFO] [06/25/2018 10:30:48.479] [RabitTracker-akka.actor.default-dispatcher-2] [akka://RabitTracker/user/Handler] [30] train-error:0.538462
[10:30:48] /xgboost/src/tree/updater_prune.cc:[10:30:48] /xgboost/src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 0 extra nodes, 274: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth= pruned nodes, max_depth=0
0
[INFO] [06/25/2018 10:30:48.639] [RabitTracker-akka.actor.default-dispatcher-9] [akka://RabitTracker/user/Handler] [31] train-error:0.538462
[10:30:48[10:30:48] /xgboost/src/tree/updater_prune.cc:] /xgboost/src/tree/updater_prune.cc:74: tree pruning end, 174: tree pruning end,  roots, 0 extra nodes, 2 pruned nodes, max_depth=0
1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
[INFO] [06/25/2018 10:30:48.799] [RabitTracker-akka.actor.default-dispatcher-4] [akka://RabitTracker/user/Handler] [32] train-error:0.538462
[10:30:48] /xgboost/src/tree/updater_prune.cc:[10:30:48] /xgboost/src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
0
[INFO] [06/25/2018 10:30:48.960] [RabitTracker-akka.actor.default-dispatcher-7] [akka://RabitTracker/user/Handler] [33] train-error:0.538462
[10:30:49] [10:30:49] /xgboost/src/tree/updater_prune.cc:/xgboost/src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
[INFO] [06/25/2018 10:30:49.119] [RabitTracker-akka.actor.default-dispatcher-2] [akka://RabitTracker/user/Handler] [34] train-error:0.538462
[10:30:49] /xgboost/src/tree/updater_prune.cc:[10:30:49] 74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
/xgboost/src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
[INFO] [06/25/2018 10:30:49.279] [RabitTracker-akka.actor.default-dispatcher-7] [akka://RabitTracker/user/Handler] [35] train-error:0.538462
[10:30:49] /xgboost/src/tree/updater_prune.cc:[10:30:49] 74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0
/xgboost/src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 0 extra nodes, 2 pruned nodes, max_depth=0

在SPark 2.1.0、2.2.0、2.3.0,Scala 2.11上进行了测试。

我正在使用xgboost的0.72版本。

我找不到我的问题的答案。我在做什么错了?

1 个答案:

答案 0 :(得分:1)

您使用Mac吗?我在Mac上的Spark xgboost遇到了完全相同的问题。

有关此的更多信息:xgboost for JVM has test failures on OS X

我手动Removed specializations for float/double parsing应用了建议的补丁程序,它起作用了。

  1. 克隆存储库
  2. 按照补丁程序的建议更改include/dmlc/parameter.h
  3. 再次编译您的xgboost代码