我已经在机器上安装了xgboost,现在我试图在Spark上运行一个简单的scala项目,尝试使用xgboost4j,但出现以下错误。
线程“主”中的异常java.lang.NoClassDefFoundError: ml / dmlc / xgboost4j / scala / spark / XGBoostClassifier
我已按照此处显示的说明进行操作 https://xgboost.readthedocs.io/en/latest/build.html 和这里 https://xgboost.readthedocs.io/en/latest/jvm/xgboost4j_spark_tutorial.html
我的项目使用sbt正确构建。
当我看着
/xgboost/jvm-packages/xgboost4j/src/main/scala/ml/dmlc/xgboost4j/scala
如scala api所示,XGBoostClassifier类不存在。
它存在于
/xgboost/jvm-packages/xgboost4j-spark/src/main/scala/ml/dmlc/xgboost4j/scala/spark
当我尝试如图所示导入时出现错误
import ml.dmlc.xgboost4j.scala.spark.XGBoostClassifier
我的build.sbt看起来像这样
name := "NewXGBoost"
version := "1.0"
scalaVersion := "2.11.8"
resolvers ++= Seq(
"GitHub Repo" at "https://raw.githubusercontent.com/CodingCat/xgboost/maven-repo/",
Resolver.mavenLocal
)
libraryDependencies += "org.apache.spark" %% "spark-core" % "2.3.2"
libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.3.2"
libraryDependencies += "org.apache.spark" %% "spark-mllib" % "2.3.2"
libraryDependencies += "org.apache.spark" %% "spark-yarn" % "2.3.2"
libraryDependencies += "ml.dmlc" % "xgboost4j-spark" % "0.82-SNAPSHOT"
libraryDependencies += "ml.dmlc" % "xgboost4j" % "0.82-SNAPSHOT"
ivyScala := ivyScala.value map { _.copy(overrideScalaVersion = true)}
期望的结果是XGBoostClassifier能够正确导入。