如何在Oozie中运行Spark程序?

时间:2018-09-25 22:53:43

标签: apache-spark oozie hue spark-java oozie-workflow

当我从Hue在Oozie编辑器中运行Spark程序时,出现以下错误:

  

日志上传时间:Tue Sep 25 19:30:03 -0300 2018     日志长度:1650年失败的Oozie Launcher,主班   [org.apache.oozie.action.hadoop.SparkMain],main()抛出异常,   org / apache / spark / deploy / SparkSubmit java.lang.NoClassDefFoundError:   org / apache / spark / deploy / Spark在以下位置提交   org.apache.oozie.action.hadoop.SparkMain.run(SparkMain.java:95)在   org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47)     在org.apache.oozie.action.hadoop.SparkMain.main(SparkMain.java:38)     在sun.reflect.NativeMethodAccessorImpl.invoke0(本机方法)处   sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)     在   sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)     在java.lang.reflect.Method.invoke(Method.java:498)在   org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:238)     在org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)处   org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)在   org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)在   org.apache.hadoop.mapred.YarnChild $ 2.run(YarnChild.java:187)在   java.security.AccessController.doPrivileged(本机方法),位于   javax.security.auth.Subject.doAs(Subject.java:422)在   org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1778)     在org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:180)     造成原因:java.lang.ClassNotFoundException:   org.apache.spark.deploy.Spark在以下位置提交   java.net.URLClassLoader.findClass(URLClassLoader.java:381)在   java.lang.ClassLoader.loadClass(ClassLoader.java:424)在   sun.misc.Launcher $ AppClassLoader.loadClass(Launcher.java:335)在   java.lang.ClassLoader.loadClass(ClassLoader.java:357)...还有16个;

enter image description here

enter image description here

0 个答案:

没有答案