我有一个带有spark依赖项的scala项目。
这些是我的依赖项：
(?=...)如果我正在运行sbt程序集，通常需要20分钟。 sbt clean和sbt test很好。它挂在包装部分。我尝试了一个非常基本的项目，同时减少了依赖性，并且组装也需要20分钟。我使用的是SBT 0.13.16和Scala 2.11.11。
请参阅我的 jstack ：
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "2.2.0" % "provided",
"org.apache.spark" %% "spark-sql" % "2.2.0",
"org.apache.spark" %% "spark-mllib" % "2.2.0",
"org.apache.spark" %% "spark-streaming-kinesis-asl" % "2.2.0",
"com.amazonaws" % "amazon-kinesis-client" % "1.8.7",
"com.amazonaws" % "aws-java-sdk-dynamodb" % "1.11.221",
"com.amazonaws" % "aws-java-sdk" % "1.11.221",
"com.amazonaws" % "aws-java-sdk-core" % "1.11.221",
"com.amazonaws" % "aws-java-sdk-sts" % "1.11.221",
"org.json4s" %% "json4s-jackson" % "3.2.11",
"org.json4s" %% "json4s-core" % "3.2.11",
"com.maxmind.geoip2" % "geoip2" % "2.9.0",
"com.maxmind.db" % "maxmind-db" % "1.2.2",
"org.uaparser" %% "uap-scala" % "0.2.0",
"org.elasticsearch" %% "elasticsearch-spark-20" % "5.6.5",
"com.google.guava" % "guava" % "23.2-jre",
"com.fasterxml.jackson.core" % "jackson-core" % "2.9.1",
"com.fasterxml.jackson.core" % "jackson-databind" % "2.9.1",
"com.fasterxml.jackson.core" % "jackson-annotations" % "2.9.1",
"com.fasterxml.jackson.dataformat" % "jackson-dataformat-cbor" % "2.9.1",
"com.fasterxml.jackson.module" % "jackson-module-scala_2.11" % "2.9.1",
"net.liftweb" %% "lift-json" % "3.1.1",
"org.scalatest" %% "scalatest" % "3.0.4" % Test,
"com.holdenkarau" %% "spark-testing-base" % "2.2.0_0.8.0" % Test
)
assemblyMergeStrategy in assembly := {
case PathList("META-INF", xs @ _*) => MergeStrategy.discard
case _ => MergeStrategy.last
}
我知道如何解决这个问题？