Question

尝试同时使用最新Apache Spark 2.0.2和Scala 2.11.7时无法解决依赖关系

我需要使用最新版本的spark，因为它支持SparkSession。以下配置适用于spark 1.6.2。但是我想要spark 2.0.*

这是我的依赖配置

scalaVersion := "2.11.7"

val sparkVersion = "2.0.2"

libraryDependencies ++= Seq(
  ws   ,
  "org.apache.spark" %% "spark-core" % sparkVersion,
  "org.apache.spark" %% "spark-sql" % sparkVersion,
  "org.apache.spark" %% "spark-streaming" % sparkVersion,
  "org.mongodb.spark" % "mongo-spark-connector_2.11" % "2.0.0",
  specs2 % Test
)

这是错误信息：

Error:Error while importing SBT project:
[info] Resolving org.ow2.asm#asm-tree;4.1 ...
[info] Resolving org.ow2.asm#asm-analysis;4.1 ...
[info] Resolving org.ow2.asm#asm-util;4.1 ...
[info] Resolving jline#jline;2.12.1 ...
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: org.glassfish.hk2#hk2-utils;2.22.2: not found
[warn]  :: org.glassfish.hk2#hk2-locator;2.22.2: not found
[warn]  :: javax.validation#validation-api;${javax.validation.version}: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[trace] Stack trace suppressed: run 'last *:update' for the full output.
[trace] Stack trace suppressed: run 'last *:ssExtractDependencies' for the full output.
[error] (*:update) sbt.ResolveException: unresolved dependency: org.glassfish.hk2#hk2-utils;2.22.2: not found
[error] unresolved dependency: org.glassfish.hk2#hk2-locator;2.22.2: not found
[error] unresolved dependency: javax.validation#validation-api;${javax.validation.version}: not found
[error] (*:ssExtractDependencies) sbt.ResolveException: unresolved dependency: org.glassfish.hk2#hk2-utils;2.22.2: not found
[error] unresolved dependency: org.glassfish.hk2#hk2-locator;2.22.2: not found
[error] unresolved dependency: javax.validation#validation-api;${javax.validation.version}: not found
[error] Total time: 81 s, completed 3 Dec, 2016 8:51:18 PM

如果我在应用依赖项时出错，请纠正我。谢谢

Answer 1

依赖性发布链中可能存在错误。一种选择是排除损坏的依赖项 - 将exclude("org.glassfish.hk2", "hk2-utils") exclude("org.glassfish.hk2", "hk2-locator") exclude("javax.validation", "validation-api")添加到引入这些依赖项的Spark依赖项中 - 并显式添加这三个依赖项的工作版本。

此外，我还没有看到org.glassfish.hk2#hk2-utils或org.glassfish.hk2#hk2-locator的版本2.22.2。

SBT：SPARK2.0。*：SCALA：未解析的依赖项org.glassfish.hk2和javax.validation

1 个答案: