我安装了cdh 5.11.0以及spark 2(根据此doc:https://www.cloudera.com/documentation/spark2/latest/topics/spark2_installing.html)。但是,当我尝试使用spark-submit2启动spark作业时,发生以下错误:
Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/impl/StaticLoggerBinder
at org.apache.spark.internal.Logging$class.initializeLogging(Logging.scala:111)
at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:102)
at org.apache.spark.deploy.yarn.ApplicationMaster$.initializeLogIfNecessary(ApplicationMaster.scala:746)
at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
at org.apache.spark.deploy.yarn.ApplicationMaster$.log(ApplicationMaster.scala:746)
at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:761)
at org.apache.spark.deploy.yarn.ExecutorLauncher$.main(ApplicationMaster.scala:795)
at org.apache.spark.deploy.yarn.ExecutorLauncher.main(ApplicationMaster.scala)
Caused by: java.lang.ClassNotFoundException: org.slf4j.impl.StaticLoggerBinder
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 8 more
我意识到类路径有问题,但是我可以用Cloudera Manager修复它吗?或者我应该在每个节点上手动进行classpath吗?