启动spark-job-server时如何修复此异常?

时间:2015-08-06 03:09:47

标签: linux scala apache-spark sbt spark-jobserver

我正在尝试在我的linux机器上启动spark-job-server。我做了以下事情:

  1. 安装了cloudera distriution CDH(5.x)并启动并运行
  2. 从上面提到的github下载spark-job-server
  3. 将项目提取到某个文件夹
  4. 已安装sbt-0.13.8并设置路径变量并能够运行sbt命令
  5. 前往项目目录(spark-job-server/)并运行sbt命令
  6. 从SBT shell中,我输入了“重新启动”,这是为了启动spark-job-server
  7. 然后,我得到了这个例外:

      ::          UNRESOLVED DEPENDENCIES         ::
    [warn]  ::::::::::::::::::::::::::::::::::::::::::::::
    [warn]  :: org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0: not found
    [warn]  :: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
    [warn]  :: org.apache.hadoop#hadoop-mapreduce-client-core;2.2.0: not found
    [warn]  :: org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.2.0: not found
    [warn]  ::::::::::::::::::::::::::::::::::::::::::::::
    [warn]
    [warn]  Note: Unresolved dependencies path:
    [warn]          org.apache.hadoop:hadoop-mapreduce-client-app:2.2.0
    [warn]            +- org.apache.hadoop:hadoop-client:2.2.0
    [warn]            +- org.apache.spark:spark-core_2.10:1.3.1 (/home/analytics/data_dir/spark-jobserver-master/project/Build.scala#L186)
    [warn]            +- spark.jobserver:job-server-api_2.10:0.5.2-SNAPSHOT
    [warn]          org.apache.hadoop:hadoop-yarn-api:2.2.0
    [warn]            +- org.apache.hadoop:hadoop-client:2.2.0
    [warn]            +- org.apache.spark:spark-core_2.10:1.3.1 (/home/analytics/data_dir/spark-jobserver-master/project/Build.scala#L186)
    [warn]            +- spark.jobserver:job-server-api_2.10:0.5.2-SNAPSHOT
    [warn]          org.apache.hadoop:hadoop-mapreduce-client-core:2.2.0
    [warn]            +- org.apache.hadoop:hadoop-client:2.2.0
    [warn]            +- org.apache.spark:spark-core_2.10:1.3.1 (/home/analytics/data_dir/spark-jobserver-master/project/Build.scala#L186)
    [warn]            +- spark.jobserver:job-server-api_2.10:0.5.2-SNAPSHOT
    [warn]          org.apache.hadoop:hadoop-mapreduce-client-jobclient:2.2.0
    [warn]            +- org.apache.hadoop:hadoop-client:2.2.0
    [warn]            +- org.apache.spark:spark-core_2.10:1.3.1 (/home/analytics/data_dir/spark-jobserver-master/project/Build.scala#L186)
    [warn]            +- spark.jobserver:job-server-api_2.10:0.5.2-SNAPSHOT
    [warn] Credentials file /root/.bintray/.credentials does not exist
    [warn] Credentials file /root/.bintray/.credentials does not exist
    [warn] Credentials file /root/.bintray/.credentials does not exist
    [trace] Stack trace suppressed: run last job-server-api/*:update for the full output.
    [trace] Stack trace suppressed: run last akka-app/*:update for the full output.
    [trace] Stack trace suppressed: run last job-server-api/*:update for the full output.
    [trace] Stack trace suppressed: run last akka-app/*:update for the full output.
    [error] (job-server-api/*:update) sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0: not found
    [error] unresolved dependency: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
    [error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-core;2.2.0: not found
    [error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.2.0: not found
    [error] (akka-app/*:update) sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0: not found
    [error] unresolved dependency: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
    [error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-core;2.2.0: not found
    [error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.2.0: not found
    [error] (job-server-api/*:update) sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0: not found
    [error] unresolved dependency: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
    [error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-core;2.2.0: not found
    [error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.2.0: not found
    [error] (akka-app/*:update) sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0: not found
    [error] unresolved dependency: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
    [error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-core;2.2.0: not found
    [error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.2.0: not found
    [error] Total time: 5 s, completed Aug 6, 2015 1:29:26 PM
    

    更多信息:

    • 我手动下载了这些依赖项并将它们放置
    • 当我再次重新运行restart命令时,sbt无法找到我在.ivy2文件夹中复制的相同依赖项

    我该如何解决这个问题?

0 个答案:

没有答案