我有一个使用Janusgraph的火花项目,我正在尝试制作一个胖罐子。这些项目在IntelliJ中运行良好,但是当我尝试制作一个胖jar并构建它并使用spark-submit运行它时,它会抛出这个错误:
NoSuchMethodError:com.google.common.base.Stopwatch.createStarted()Lcom/google/common/base/Stopwatch
我正在使用sbt clean compile
和sbt assembly
创建一个胖罐。
我还解压缩了创建的jar并查找了Stopwatch.class和提到的方法。他们似乎在那里。所以,我不明白为什么会出现这个错误。
我的build.sbt
文件如下:
name := "proj1"
version := "0.1"
scalaVersion := "2.11.12"
// https://mvnrepository.com/artifact/com.michaelpollmeier/gremlin-scala
libraryDependencies += "com.michaelpollmeier" %% "gremlin-scala" % "3.3.1.1"
// https://mvnrepository.com/artifact/org.janusgraph/janusgraph-core
libraryDependencies += "org.janusgraph" % "janusgraph-core" % "0.2.0"
// https://mvnrepository.com/artifact/org.apache.spark/spark-core
libraryDependencies += "org.apache.spark" %% "spark-core" % "2.3.0"
// https://mvnrepository.com/artifact/org.apache.spark/spark-sql
libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.3.0"
// https://mvnrepository.com/artifact/org.apache.spark/spark-mllib
libraryDependencies += "org.apache.spark" %% "spark-mllib" % "2.3.0"
// https://mvnrepository.com/artifact/org.apache.spark/spark-hive
libraryDependencies += "org.apache.spark" %% "spark-hive" % "2.3.0"
//libraryDependencies += "com.google.guava" %% "guava" % "15.0"
//libraryDependencies += "org.apache.hadoop" %% "hadoop-client" % "2.7.2"
// https://mvnrepository.com/artifact/org.janusgraph/janusgraph-es
libraryDependencies += "org.janusgraph" % "janusgraph-es" % "0.2.0"
// https://mvnrepository.com/artifact/org.janusgraph/janusgraph-cassandra
libraryDependencies += "org.janusgraph" % "janusgraph-cassandra" % "0.2.0"
// https://mvnrepository.com/artifact/com.google.guava/guava
libraryDependencies += "com.google.guava" % "guava" % "24.0-jre"
// https://mvnrepository.com/artifact/commons-io/commons-io
libraryDependencies += "commons-io" % "commons-io" % "2.6"
assemblyMergeStrategy in assembly := {
case PathList("META-INF", xs @ _*) => MergeStrategy.discard
case x => MergeStrategy.first
}
感谢任何帮助。提前谢谢!