Question

我尝试使用scalatest执行单元测试，利用hbase测试实用程序在本地测试开发代码。 sbt中hbase测试实用程序的设置是现在的斗争。当我编译时，我收到以下错误：

[warn]  module not found: org.apache.hbase#${compat.module};1.2.1
[warn] ==== local: tried
[warn]   /root/.ivy2/local/org.apache.hbase/${compat.module}/1.2.1/ivys/ivy.xml
[warn] ==== public: tried
[warn]   https://repo1.maven.org/maven2/org/apache/hbase/${compat.module}/1.2.1/${compat.module}-1.2.1.pom
[warn] ==== Akka Repository: tried
[warn]   http://repo.akka.io/releases/org/apache/hbase/${compat.module}/1.2.1/${compat.module}-1.2.1.pom
[warn] ==== scala-tools: tried
[warn]   https://oss.sonatype.org/content/groups/scala-tools/org/apache/hbase/${compat.module}/1.2.1/${compat.module}-1.2.1.pom
[warn] ==== cloudera-repos: tried
[warn]   https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/hbase/${compat.module}/1.2.1/${compat.module}-1.2.1.pom
[warn] ==== Sonatype OSS Snapshots: tried
[warn]   https://oss.sonatype.org/content/repositories/snapshots/org/apache/hbase/${compat.module}/1.2.1/${compat.module}-1.2.1.pom
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: org.apache.hbase#${compat.module};1.2.1: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn] 
[warn]  Note: Unresolved dependencies path:
[warn]      org.apache.hbase:${compat.module}:1.2.1
[warn]        +- org.apache.hbase:hbase-testing-util:1.2.1 (/workspace/spark/etl/built.sbt#L30-62)

[trace] Stack trace suppressed: run last *:update for the full output.
[error] (*:update) sbt.ResolveException: unresolved dependency: org.apache.hbase#${compat.module};1.2.1: not found
[error] Total time: 32 s, completed Apr 29, 2016 9:25:27 AM

我的build.sbt文件如下：

val hbaseVersion = "1.2.1"
val sparkVersion = "1.6.1"
val hadoopVersion = "2.7.1"

libraryDependencies ++= Seq(
  "org.apache.spark" %% "spark-core" % sparkVersion % "provided",
  "org.apache.spark" %% "spark-streaming" % sparkVersion % "provided",
  "org.apache.spark" %% "spark-streaming-kafka" % sparkVersion,
  "org.apache.spark" %% "spark-sql" % sparkVersion % "provided",
  "org.apache.spark" %% "spark-mllib" % sparkVersion ,
  "org.apache.hbase" % "hbase" % hbaseVersion,
  "org.apache.hbase" % "hbase-server" % hbaseVersion,
  "org.apache.hbase" % "hbase-server" % hbaseVersion classifier "tests",
  "org.apache.hbase" % "hbase-client" % hbaseVersion,
  "org.apache.hbase" % "hbase-common" % hbaseVersion,
  "org.apache.hbase" % "hbase-common" % hbaseVersion classifier "tests",
  "org.apache.hbase" % "hbase-annotations" % hbaseVersion,
  "org.apache.hbase" % "hbase-testing-util" % hbaseVersion % "test",
  "org.apache.hadoop" % "hadoop-minicluster" % hadoopVersion,
  "org.apache.hadoop" % "hadoop-mapreduce-client-jobclient" % hadoopVersion classifier "tests",
  "org.apache.hadoop" % "hadoop-hdfs" % hadoopVersion,
  "org.apache.hadoop" % "hadoop-hdfs" % hadoopVersion classifier "tests",
  "org.apache.hbase" % "hbase-hadoop-compat" % hbaseVersion,
  "org.apache.hbase" % "hbase-hadoop-compat" % hbaseVersion classifier "tests",
  "org.apache.hbase" % "hbase-hadoop2-compat" % hbaseVersion,
  "org.apache.hbase" % "hbase-hadoop2-compat" % hbaseVersion classifier "tests",
  "org.apache.hadoop" % "hadoop-common" % hadoopVersion,
  "org.apache.hadoop" % "hadoop-common" % hadoopVersion classifier "tests",
  "org.apache.hadoop" % "hadoop-annotations" % hadoopVersion,
  "org.scalatest" %% "scalatest" % "2.2.6" % "test" ,
  //"org.scalacheck" %% "scalacheck" % "1.12.5" % "test",
  "com.cloudera.sparkts" % "sparkts" % "0.3.0",
  "com.ecwid.consul" % "consul-api" % "1.1.9",
  "joda-time" % "joda-time" % "2.7"
)

resolvers ++= Seq(
  "Akka Repository" at "http://repo.akka.io/releases/",
  "scala-tools" at "https://oss.sonatype.org/content/groups/scala-tools",
  "cloudera-repos" at "https://repository.cloudera.com/artifactory/cloudera-repos/",
  "Sonatype OSS Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots"
)

任何人都明白为什么会发生这种失败？

Answer 1

抱歉延迟回复。我不能让它按原样运行，所以我改变了这样的版本：

val sparkVersion = "1.6.1"
val hbaseVersion = "1.2.0-cdh5.7.0"
val hadoopVersion = "2.6.0-cdh5.7.0"

这导致了更多的麻烦。由于引用了较旧的库方法，我不得不将番石榴的版本更改为早期版本，因此这是必需的：

"com.google.guava" % "guava" % "14.0" force()

（我认为最高版本16.0很好）

不得不评论以下内容：

// "com.cloudera" % "spark-hbase" % "0.0.2-clabs",

（原来的q不存在）

最后，看起来原始问题是需要解决的问题，请参阅此处，感谢David Portabella的参考：

https://issues.apache.org/jira/browse/HBASE-15925

并修正了1.2.2版本

使用hbase-testing-utility进行sbt依赖管理问题

1 个答案: