Question

我正在尝试安装Apache Spark。执行./sbt/sbt assembly命令后

以下是错误：

Resolving org.eclipse.paho#mqtt-client;0.4.0 ...
[error] Server access Error: Connection refused                                                                              
url=https://oss.sonatype.org/content/repositories/snapshots/org/eclipse/paho/mqtt- client/0.4.0/mqtt-client-0.4.0.pom
[error] Server access Error: Connection refused             
url=https://oss.sonatype.org/service/local/staging/deploy/maven2/org/eclipse/paho/mqtt-client/0.4.0/mqtt-client-0.4.0.pom

[warn]  module not found: org.eclipse.paho#mqtt-client;0.4.0
[warn] ==== local: tried
[warn]   /home/raghuveer/.ivy2/local/org.eclipse.paho/mqtt-client/0.4.0/ivys/ivy.xml
[warn] ==== Local Maven Repo: tried
[warn] ==== sonatype-snapshots: tried

[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: org.eclipse.paho#mqtt-client;0.4.0: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[info] Resolving org.apache.spark#spark-core_2.10;0.9.0-incubating ...

UNRESOLVED DEPENDENCIES
org.eclipse.paho#mqtt-client;0.4.0: not found

sbt.ResolveException：未解析的依赖项：org.eclipse.paho＃mqtt-client; 0.4.0：未找到

build.properties sbt.version = 0.12.4

sparkBuild.scala scalaVersion：=“2.10.3”

Answer 1

免责声明这还不是一个完整的答案......在OP解决问题之前，预计会有更多更新，直到达到最终版本阶段。

可能存在许多因素阻止项目的依赖性得到解决，项目最终会被组装。这是一个确定未解决依赖的根本原因的方法：org.eclipse.paho＃mqtt-client; 0.4.0：not found issue。

转到项目的主目录，说~/apps/spark并执行./sbt/sbt。

jacek:~/apps/spark
$ ./sbt/sbt
Launching sbt from sbt/sbt-launch-0.12.4.jar
[info] Loading global plugins from /Users/jacek/.sbt/plugins
[info] Loading project definition from /Users/jacek/apps/spark-0.9.0-incubating/project/project
[info] Loading project definition from /Users/jacek/apps/spark-0.9.0-incubating/project
[info] Set current project to root (in build file:/Users/jacek/apps/spark-0.9.0-incubating/)
>

在shell中键入projects以列出Spark下定义的所有项目。

> projects
[info] In file:/Users/jacek/apps/spark-0.9.0-incubating/
[info]     assembly
[info]     bagel
[info]     core
[info]     examples
[info]     external-flume
[info]     external-kafka
[info]     external-mqtt
[info]     external-twitter
[info]     external-zeromq
[info]     graphx
[info]     mllib
[info]     repl
[info]   * root
[info]     streaming
[info]     tools
[info]     yarn
[info]     yarn-alpha

将当前项目（标有星号*）更改为依赖于org.eclipse.paho#mqtt-client;0.4.0 external-mqtt的模块。

> project external-mqtt
[info] Set current project to spark-streaming-mqtt (in build file:/Users/jacek/apps/spark-0.9.0-incubating/)

show项目的依赖项。

> show library-dependencies
[info] List(org.scala-lang:scala-library:2.10.3, io.netty:netty-all:4.0.13.Final, org.eclipse.jetty:jetty-server:7.6.8.v20121106, org.eclipse.jetty.orbit:javax.servlet:2.5.0.v201103041518, org.scalatest:scalatest:1.9.1:test, org.scalacheck:scalacheck:1.10.0:test, com.novocode:junit-interface:0.9:test, org.easymock:easymock:3.1:test, org.mockito:mockito-all:1.8.5:test, commons-io:commons-io:2.4:test, commons-io:commons-io:2.4, org.eclipse.paho:mqtt-client:0.4.0)

请注意列表中的最后一个依赖项org.eclipse.paho:mqtt-client:0.4.0。

执行update以解决依赖关系（我希望我将要求您添加到问题中的依赖项解析错误）。

> update
[info] Updating {file:/Users/jacek/apps/spark-0.9.0-incubating/}external-mqtt...
[info] Resolving org.mockito#mockito-all;1.8.5 ...
[info] Done updating.
[success] Total time: 24 s, completed Feb 15, 2014 3:02:27 PM

正如您所注意到的，它对我来说很好。 update为external/mqtt/target/resolution-cache/reports目录中项目的每个配置写了几个解析报告。请注意，由于您的解决方案失败，因此可能很少或根本没有报告。

由于org.eclipse.paho:mqtt-client:0.4.0依赖关系位于compile范围内，请在浏览器中打开org.apache.spark-spark-streaming-mqtt_2.10-compile.xml报告（在MacOS上我必须使用Safari，因为Chrome不起作用 - 不知道其他操作系统）。

在报告中应该有mqtt-client by org.eclipse.paho依赖项的部分。

enter image description here

但是我怀疑你在目录中找到了任何报告，因为update在你的情况下仍然失败。

为了找到确定根本原因的方法，我在Spark的project/SparkBuild.scala构建配置中输入了一个拼写错误，用于存在依赖关系的Eclipse Repo存储库的URL，并从我当地的常春藤中删除了依赖关系repo（它位于文件的最后 - 在mqttSettings def中 - 为什么这是def而不是lazy val是另一个问题，但我离题了）

jacek:~/apps/spark
$ rm -rf .ivy2/cache/org.eclipse.paho/mqtt-client

我重新运行./sbt/sbt。

jacek:~/apps/spark
$ ./sbt/sbt
Launching sbt from sbt/sbt-launch-0.12.4.jar
[info] Loading global plugins from /Users/jacek/.sbt/plugins
[info] Loading project definition from /Users/jacek/apps/spark-0.9.0-incubating/project/project
[info] Loading project definition from /Users/jacek/apps/spark-0.9.0-incubating/project
[info] Compiling 1 Scala source to /Users/jacek/apps/spark-0.9.0-incubating/project/target/scala-2.9.2/sbt-0.12/classes...
[info] Set current project to root (in build file:/Users/jacek/apps/spark-0.9.0-incubating/)

在SBT shell中，执行external-mqtt/update以启动update项目中的external-mqtt任务。

> external-mqtt/update
[info] Updating {file:/Users/jacek/apps/spark-0.9.0-incubating/}external-mqtt...
[info] Resolving commons-codec#commons-codec;1.3 ...
[warn] Host repoo.eclipse.org not found. url=https://repoo.eclipse.org/content/repositories/paho-releases/commons-codec/commons-codec/1.3/commons-codec-1.3.pom
[info] You probably access the destination server through a proxy server that is not well configured.
[info] Resolving commons-collections#commons-collections;3.2.1 ...
[warn] Host repoo.eclipse.org not found. url=https://repoo.eclipse.org/content/repositories/paho-releases/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.pom
[info] You probably access the destination server through a proxy server that is not well configured.
[info] Resolving commons-lang#commons-lang;2.4 ...
[warn] Host repoo.eclipse.org not found. url=https://repoo.eclipse.org/content/repositories/paho-releases/commons-lang/commons-lang/2.4/commons-lang-2.4.pom
[info] You probably access the destination server through a proxy server that is not well configured.
[info] Resolving oro#oro;2.0.8 ...
[warn] Host repoo.eclipse.org not found. url=https://repoo.eclipse.org/content/repositories/paho-releases/oro/oro/2.0.8/oro-2.0.8.pom
[info] You probably access the destination server through a proxy server that is not well configured.
[info] Resolving commons-el#commons-el;1.0 ...
[warn] Host repoo.eclipse.org not found. url=https://repoo.eclipse.org/content/repositories/paho-releases/commons-el/commons-el/1.0/commons-el-1.0.pom
[info] You probably access the destination server through a proxy server that is not well configured.
[info] Resolving hsqldb#hsqldb;1.8.0.10 ...
[warn] Host repoo.eclipse.org not found. url=https://repoo.eclipse.org/content/repositories/paho-releases/hsqldb/hsqldb/1.8.0.10/hsqldb-1.8.0.10.pom
[info] You probably access the destination server through a proxy server that is not well configured.
[info] Resolving org.eclipse.paho#mqtt-client;0.4.0 ...
[warn] Host repoo.eclipse.org not found. url=https://repoo.eclipse.org/content/repositories/paho-releases/org/eclipse/paho/mqtt-client/0.4.0/mqtt-client-0.4.0.pom
[info] You probably access the destination server through a proxy server that is not well configured.
[warn]  module not found: org.eclipse.paho#mqtt-client;0.4.0
[warn] ==== local: tried
[warn]   /Users/jacek/.ivy2/local/org.eclipse.paho/mqtt-client/0.4.0/ivys/ivy.xml
[warn] ==== Local Maven Repo: tried
[warn] ==== sonatype-snapshots: tried
[warn]   https://oss.sonatype.org/content/repositories/snapshots/org/eclipse/paho/mqtt-client/0.4.0/mqtt-client-0.4.0.pom
[warn] ==== sonatype-staging: tried
[warn]   https://oss.sonatype.org/service/local/staging/deploy/maven2/org/eclipse/paho/mqtt-client/0.4.0/mqtt-client-0.4.0.pom
[warn] ==== Eclipse Repo: tried
[warn]   https://repoo.eclipse.org/content/repositories/paho-releases/org/eclipse/paho/mqtt-client/0.4.0/mqtt-client-0.4.0.pom
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/org/eclipse/paho/mqtt-client/0.4.0/mqtt-client-0.4.0.pom
[info] Resolving org.mockito#mockito-all;1.8.5 ...
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: org.eclipse.paho#mqtt-client;0.4.0: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[trace] Stack trace suppressed: run last external-mqtt/*:update for the full output.
[error] (external-mqtt/*:update) sbt.ResolveException: unresolved dependency: org.eclipse.paho#mqtt-client;0.4.0: not found
[error] Total time: 21 s, completed Feb 15, 2014 3:16:48 PM

显然失败了（因为https://repoo.eclipse.org中存在拼写错误 - 请注意双o。

这是输出我要求您添加到您的问题中，以便更容易为您提供帮助。

如果您想了解SBT如何解析依赖关系，请使用last update - 有关该流程的大量信息，不是对于胆小的人。

SBT报告未解决的依赖关系：org.eclipse.paho＃mqtt-client;安装Apache Spark时为0.4.0

1 个答案: