sbt assembly命令显示错误:(redshiftConnector / *:assembly)deduplicate:找到不同的文件内容

时间:2015-10-01 12:42:07

标签: scala sbt sbt-0.13

sbt程序集给了我错误我正在尝试使用此链接创建一个胖jar,其标题为Exclude特定传递deps,它们排除了导致此问题的一些库但在我的项目build.sbt中文件没有依赖是在sbt控制台上显示。 这是堆栈跟踪

[info] Including from cache: scala-library-2.11.7.jar
[info] Checking every *.class/*.jar file's SHA-1.
[info] Merging files...
[warn] Merging 'META-INF/MANIFEST.MF' with strategy 'discard'
[warn] Strategy 'discard' was applied to a file
[info] Assembly up to date: /home/user/Downloads/ScalaProj/datasembly/target/scala-2.11/datasembly-assembly-0.1-SNAPSHOT.jar
[info] Including from cache: jackson-datatype-jsr310-2.5.4.jar
[info] Including from cache: scala-library-2.11.7.jar
[info] Including from cache: httpcore-4.3.2.jar
[info] Including from cache: postgresql-9.4-1201-jdbc41.jar
[info] Including from cache: commons-codec-1.6.jar
[info] Including from cache: slf4j-nop-1.6.4.jar
[info] Including from cache: amazon-kinesis-producer-0.10.1.jar
[info] Including from cache: joda-time-2.8.2.jar
[info] Including from cache: HikariCP-2.3.8.jar
[info] Including from cache: chill_2.11-0.7.0.jar
[info] Including from cache: chill-java-0.7.0.jar
[info] Including from cache: javassist-3.18.2-GA.jar
[info] Including from cache: kryo-2.21.jar
[info] Including from cache: guava-18.0.jar
[info] Including from cache: slick-pg_2.11-0.9.1.jar
[info] Including from cache: reflectasm-1.07-shaded.jar
[info] Including from cache: slf4j-api-1.7.12.jar
[info] Including from cache: asm-4.0.jar
[info] Including from cache: slick-pg_core_2.11-0.9.1.jar
[info] Including from cache: minlog-1.2.jar
[info] Including from cache: protobuf-java-2.6.1.jar
[info] Including from cache: slf4j-simple-1.7.12.jar
[info] Including from cache: objenesis-1.2.jar
[info] Including from cache: commons-io-2.4.jar
[info] Including from cache: flyway-core-3.2.1.jar
[info] Including from cache: commons-lang-2.6.jar
[info] Including from cache: aws-java-sdk-core-1.9.37.jar
[info] Including from cache: commons-logging-1.1.3.jar
[info] Including from cache: play-json_2.11-2.4.2.jar
[info] Including from cache: play-iteratees_2.11-2.4.2.jar
[info] Including from cache: httpclient-4.3.4.jar
[info] Including from cache: play-datacommons_2.11-2.4.2.jar
[info] Including from cache: joda-convert-1.7.jar
[info] Including from cache: scala-stm_2.11-0.7.jar
[info] Including from cache: scala-reflect-2.11.6.jar
[info] Including from cache: config-1.3.0.jar
[info] Including from cache: jackson-core-2.5.4.jar
[info] Including from cache: jackson-annotations-2.5.4.jar
[info] Including from cache: slick_2.11-3.0.0.jar
[info] Including from cache: play-functional_2.11-2.4.2.jar
[info] Including from cache: reactive-streams-1.0.0.jar
[info] Including from cache: jackson-databind-2.5.4.jar
[info] Including from cache: jackson-datatype-jdk8-2.5.4.jar
[info] Checking every *.class/*.jar file's SHA-1.
[info] Merging files...
[warn] Merging 'META-INF/DEPENDENCIES' with strategy 'discard'
[warn] Merging 'META-INF/INDEX.LIST' with strategy 'discard'
[warn] Merging 'META-INF/MANIFEST.MF' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.amazonaws/amazon-kinesis-producer/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.amazonaws/amazon-kinesis-producer/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.amazonaws/aws-java-sdk-core/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.amazonaws/aws-java-sdk-core/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.esotericsoftware.kryo/kryo/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.esotericsoftware.kryo/kryo/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.esotericsoftware.minlog/minlog/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.esotericsoftware.minlog/minlog/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.esotericsoftware.reflectasm/reflectasm/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.esotericsoftware.reflectasm/reflectasm/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.fasterxml.jackson.core/jackson-annotations/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.fasterxml.jackson.core/jackson-annotations/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.fasterxml.jackson.core/jackson-core/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.fasterxml.jackson.core/jackson-core/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.fasterxml.jackson.core/jackson-databind/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.fasterxml.jackson.core/jackson-databind/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.fasterxml.jackson.datatype/jackson-datatype-jdk8/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.fasterxml.jackson.datatype/jackson-datatype-jdk8/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.fasterxml.jackson.datatype/jackson-datatype-jsr310/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.fasterxml.jackson.datatype/jackson-datatype-jsr310/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.google.guava/guava/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.google.guava/guava/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.google.protobuf/protobuf-java/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.google.protobuf/protobuf-java/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.zaxxer/HikariCP/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/com.zaxxer/HikariCP/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/commons-codec/commons-codec/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/commons-codec/commons-codec/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/commons-io/commons-io/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/commons-io/commons-io/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/commons-lang/commons-lang/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/commons-lang/commons-lang/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/commons-logging/commons-logging/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/commons-logging/commons-logging/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/joda-time/joda-time/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/joda-time/joda-time/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/org.apache.httpcomponents/httpclient/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/org.apache.httpcomponents/httpclient/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/org.apache.httpcomponents/httpcore/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/org.apache.httpcomponents/httpcore/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/org.flywaydb/flyway-core/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/org.flywaydb/flyway-core/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/org.javassist/javassist/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/org.javassist/javassist/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/org.joda/joda-convert/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/org.joda/joda-convert/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/org.slf4j/slf4j-api/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/org.slf4j/slf4j-api/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/org.slf4j/slf4j-nop/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/org.slf4j/slf4j-nop/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/maven/org.slf4j/slf4j-simple/pom.properties' with strategy 'discard'
[warn] Merging 'META-INF/maven/org.slf4j/slf4j-simple/pom.xml' with strategy 'discard'
[warn] Merging 'META-INF/services/com.fasterxml.jackson.databind.Module' with strategy 'filterDistinctLines'
[error] 5 errors were encountered during merge
                                                                        [info] Checking every *.class/*.jar file's SHA-1.
[info] Merging files...
[info] Assembly up to date: /home/user/Downloads/ScalaProj/datasembly/galactus/target/scala-2.11/galactus-assembly-0.1.0.jar
[trace] Stack trace suppressed: run last redshiftConnector/*:assembly for the full output.
[error] (redshiftConnector/*:assembly) deduplicate: different file contents found in the following:
[error] /home/user/.ivy2/cache/com.esotericsoftware.kryo/kryo/bundles/kryo-2.21.jar:com/esotericsoftware/minlog/Log$Logger.class
[error] /home/user/.ivy2/cache/com.esotericsoftware.minlog/minlog/jars/minlog-1.2.jar:com/esotericsoftware/minlog/Log$Logger.class
[error] deduplicate: different file contents found in the following:
[error] /home/user/.ivy2/cache/com.esotericsoftware.kryo/kryo/bundles/kryo-2.21.jar:com/esotericsoftware/minlog/Log.class
[error] /home/user/.ivy2/cache/com.esotericsoftware.minlog/minlog/jars/minlog-1.2.jar:com/esotericsoftware/minlog/Log.class
[error] deduplicate: different file contents found in the following:
[error] /home/user/.ivy2/cache/org.slf4j/slf4j-nop/jars/slf4j-nop-1.6.4.jar:org/slf4j/impl/StaticLoggerBinder.class
[error] /home/user/.ivy2/cache/org.slf4j/slf4j-simple/jars/slf4j-simple-1.7.12.jar:org/slf4j/impl/StaticLoggerBinder.class
[error] deduplicate: different file contents found in the following:
[error] /home/user/.ivy2/cache/org.slf4j/slf4j-nop/jars/slf4j-nop-1.6.4.jar:org/slf4j/impl/StaticMDCBinder.class
[error] /home/user/.ivy2/cache/org.slf4j/slf4j-simple/jars/slf4j-simple-1.7.12.jar:org/slf4j/impl/StaticMDCBinder.class
[error] deduplicate: different file contents found in the following:
[error] /home/user/.ivy2/cache/org.slf4j/slf4j-nop/jars/slf4j-nop-1.6.4.jar:org/slf4j/impl/StaticMarkerBinder.class
[error] /home/user/.ivy2/cache/org.slf4j/slf4j-simple/jars/slf4j-simple-1.7.12.jar:org/slf4j/impl/StaticMarkerBinder.class
[error] Total time: 41 s, completed Oct 1, 2015 5:21:47 PM

这是我的build.sbt文件

import com.amazonaws.services.s3.model.Region
import ohnosequences.sbt.SbtS3Resolver.autoImport._

name := "h"

lazy val commonSettings = Seq(
  organization := "datasembly",
  version := "0.1.0-SNAPSHOT",
  scalaVersion := "2.11.7",
  resolvers += "akka" at "http://repo.akka.io/snapshots",
  resolvers += Resolver.sonatypeRepo("public"),
  publishMavenStyle := false,
  s3region := Region.US_Standard,
  awsProfile := "maven",
  publishTo := {
    val prefix = if (isSnapshot.value) "snapshots" else "releases"
    Some(s3resolver.value("My "+prefix+" S3 bucket", s3(prefix+".mvn-repo.d")) withIvyPatterns)
  }
)

lazy val commonDepends = Seq(
  libraryDependencies ++= Seq(
    "com.github.scopt"            %%  "scopt"                     %   "3.3.0"
  )
)

lazy val clusterDepends = Seq(
  libraryDependencies ++= Seq(
    "com.typesafe.akka"       %%  "akka-cluster"      %   "2.4-M2",
    "com.twitter"             %%  "chill-all"         %   "0.7.0",
    "com.twitter"             %%  "chill-akka"        %   "0.7.0"
  )
)

lazy val root = (project in file(".")) aggregate(core, worker, master, slick, aws) settings(commonSettings: _*)

lazy val core = project settings(commonSettings: _*) settings(clusterDepends: _*)

lazy val worker = project dependsOn core settings(commonSettings: _*) settings(commonDepends: _*) settings(clusterDepends: _*) enablePlugins JavaAppPackaging enablePlugins UniversalPlugin

lazy val master = project dependsOn core settings(commonSettings: _*) settings(commonDepends: _*) settings(clusterDepends: _*) enablePlugins JavaAppPackaging

lazy val client = project dependsOn core settings(commonSettings: _*) settings(commonDepends: _*) enablePlugins JavaAppPackaging

// Extensions

lazy val aws = project in file("extensions/aws") settings(commonSettings: _*) dependsOn core

// Contrib

lazy val slick = project in file("contrib/slick") settings(commonSettings: _*) dependsOn core

bulid.sbt

name := "h-core"

libraryDependencies ++= Seq(
  "org.apache.commons"          %   "commons-compress"          %   "1.9",
  "com.jsuereth"                %%  "scala-arm"                 %   "1.4"
)

build.sbt

name := "h-client"

libraryDependencies ++= Seq(
  "jline"                       %   "jline"           %   "2.12",
  "com.typesafe.play"           %%  "play-ws"         %   "2.4.1"
)

build.sbt     name:=“h-master”

libraryDependencies ++= Seq(
  "com.typesafe.play"           %%    "play"                      %   "2.4.1",
  "com.typesafe.play"           %%    "play-netty-server"         %   "2.4.1",
  "com.leansoft"                %     "bigqueue"                  %   "0.7.0"
)

resolvers += "github.release.repo" at "https://raw.githubusercontent.com/bulldog2011/bulldog-repo/master/repo/releases/"

checksums in update := Nil

更新2尝试使用我的用户rosshsr解决方案,但错误保持不变

libraryDependencies +=  excludeAll(

  ExclusionRule("com.esotericsoftware.minlog" , "minlog"),
  ExclusionRule("org.slf4j" ,   "slf4j-nop"),
  ExclusionRule("com.esotericsoftware.kryo" % "kryo")

)

请帮忙

1 个答案:

答案 0 :(得分:1)

由于我不熟悉所有这些库中的哪个位置,这取决于这些冲突的发生,我只是给你我解决这个问题的一般策略。

您需要排除其中一个冲突依赖项。

libraryDependencies += "org.apache.hadoop" % "hadoop-common" % "2.0.0-alpha" excludeAll(

  ExclusionRule("commons-logging","commons-logging-api"),
  ExclusionRule("asm","asm")

)

这是我的一个build.sbt文件的摘录。

您需要将excludeAll位附加到依赖项并添加ExclusionRule("org.slf4j","slf4j-simple")

之类的内容