下载包

时间:2016-11-15 01:48:49

标签: scala apache-spark apache-spark-sql apache-spark-mllib

我在程序中使用了sparkts。它在本地运行成功。

但是,当我使用

将jar部署到远程集群时
spark-submit --packages com.cloudera.sparkts:sparkts:0.4.0  myjar.jar

它将在线搜索sparkTS包,但是失败并返回错误。我不知道发生了什么。我哪里错了?

Ivy Default Cache set to: /home/myname/.ivy2/cache

The jars for the packages stored in: /home/myname/.ivy2/jars

:: loading settings :: url = jar:file:/data1/opt/spark/spark-1.6.1/lib/spark-assembly-1.6.1-hadoop2.7.0-mapr-1506.jar!/org/apache/ivy/core/settings/ivysettings.xml

com.cloudera.sparkts#sparkts added as a dependency

:: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0

confs: [default]

found com.cloudera.sparkts#sparkts;0.4.0 in central

found org.scala-lang#scalap;2.11.8 in central

found org.scala-lang#scala-compiler;2.11.8 in central

found org.scala-lang#scala-reflect;2.11.8 in central

found org.scala-lang.modules#scala-xml_2.11;1.0.4 in central

found org.scala-lang.modules#scala-parser-combinators_2.11;1.0.4 in central

found org.threeten#threeten-extra;0.9 in central

found org.scalanlp#breeze_2.11;0.12 in central

found org.scalanlp#breeze-macros_2.11;0.12 in central

found com.github.fommil.netlib#core;1.1.2 in central

found net.sourceforge.f2j#arpack_combined_all;0.1 in central

found net.sf.opencsv#opencsv;2.3 in local-m2-cache

found com.github.rwl#jtransforms;2.4.0 in central

found org.spire-math#spire_2.11;0.7.4 in central

found org.spire-math#spire-macros_2.11;0.7.4 in central

found org.slf4j#slf4j-api;1.7.5 in local-m2-cache

found com.chuusai#shapeless_2.11;2.0.0 in central

found org.scalanlp#breeze-viz_2.11;0.12 in central

found jfree#jcommon;1.0.16 in central

found jfree#jfreechart;1.0.13 in central

found org.apache.xmlgraphics#xmlgraphics-commons;1.3.1 in central

found commons-io#commons-io;1.3.1 in central

found commons-logging#commons-logging;1.0.4 in local-m2-cache

found com.lowagie#itext;2.1.5 in central

found bouncycastle#bcmail-jdk14;138 in central

found bouncycastle#bcprov-jdk14;138 in central

found org.apache.commons#commons-math3;3.4.1 in local-m2-cache

:: resolution report :: resolve 3308ms :: artifacts dl 100ms

:: modules in use:

bouncycastle#bcmail-jdk14;138 from central in [default]

bouncycastle#bcprov-jdk14;138 from central in [default]

com.chuusai#shapeless_2.11;2.0.0 from central in [default]

com.cloudera.sparkts#sparkts;0.4.0 from central in [default]

com.github.fommil.netlib#core;1.1.2 from central in [default]

com.github.rwl#jtransforms;2.4.0 from central in [default]

com.lowagie#itext;2.1.5 from central in [default]

commons-io#commons-io;1.3.1 from central in [default]

commons-logging#commons-logging;1.0.4 from local-m2-cache in [default]

jfree#jcommon;1.0.16 from central in [default]

jfree#jfreechart;1.0.13 from central in [default]

net.sf.opencsv#opencsv;2.3 from local-m2-cache in [default]

net.sourceforge.f2j#arpack_combined_all;0.1 from central in [default]

org.apache.commons#commons-math3;3.4.1 from local-m2-cache in [default]

org.apache.xmlgraphics#xmlgraphics-commons;1.3.1 from central in [default]

org.scala-lang#scala-compiler;2.11.8 from central in [default]

org.scala-lang#scala-reflect;2.11.8 from central in [default]

org.scala-lang#scalap;2.11.8 from central in [default]

org.scala-lang.modules#scala-parser-combinators_2.11;1.0.4 from central in [default]

org.scala-lang.modules#scala-xml_2.11;1.0.4 from central in [default]

org.scalanlp#breeze-macros_2.11;0.12 from central in [default]

org.scalanlp#breeze-viz_2.11;0.12 from central in [default]

org.scalanlp#breeze_2.11;0.12 from central in [default]

org.slf4j#slf4j-api;1.7.5 from local-m2-cache in [default]

org.spire-math#spire-macros_2.11;0.7.4 from central in [default]

org.spire-math#spire_2.11;0.7.4 from central in [default]

org.threeten#threeten-extra;0.9 from central in [default]

:: evicted modules:

org.scala-lang#scala-reflect;2.11.7 by [org.scala-lang#scala-reflect;2.11.8] in [default]

org.scala-lang#scala-reflect;2.11.0 by [org.scala-lang#scala-reflect;2.11.8] in [default]

---------------------------------------------------------------------

|                  |            modules            ||   artifacts   |

|       conf       | number| search|dwnlded|evicted|| number|dwnlded|

---------------------------------------------------------------------

|      default     |   29  |   0   |   0   |   2   ||   27  |   0   |

---------------------------------------------------------------------



:: problems summary ::

:::: WARNINGS

[NOT FOUND  ] org.slf4j#slf4j-api;1.7.5!slf4j-api.jar (4ms)



==== local-m2-cache: tried



  file:/home/myname/.m2/repository/org/slf4j/slf4j-api/1.7.5/slf4j-api-1.7.5.jar



[NOT FOUND  ] commons-logging#commons-logging;1.0.4!commons-logging.jar (1ms)



==== local-m2-cache: tried



  file:/home/myname/.m2/repository/commons-logging/commons-logging/1.0.4/commons-logging-1.0.4.jar



::::::::::::::::::::::::::::::::::::::::::::::



::              FAILED DOWNLOADS            ::



:: ^ see resolution messages for details  ^ ::



::::::::::::::::::::::::::::::::::::::::::::::



:: org.slf4j#slf4j-api;1.7.5!slf4j-api.jar



:: commons-logging#commons-logging;1.0.4!commons-logging.jar



::::::::::::::::::::::::::::::::::::::::::::::







:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS

Exception in thread "main" java.lang.RuntimeException: [download failed: org.slf4j#slf4j-api;1.7.5!slf4j-api.jar, download failed: commons-logging#commons-logging;1.0.4!commons-logging.jar]

at org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1072)

at org.apache.spark.deploy.SparkSubmit$.prepareSubmitEnvironment(SparkSubmit.scala:287)

at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:154)

at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)

at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

0 个答案:

没有答案