我无法提交只执行System.out.println()的简单作业。这是我从SnappyData Lead返回的错误。
snappy-job.sh submit --lead 10.0.18.66:8090 --app-name SimpleJobApp
--class snappydata.jobs.SimpleJob --app-jar simpleJob.jar OKOK{ "status": "ERROR", "result": {
"message": "null",
"errorClass": "scala.MatchError",
"stack": ["spark.jobserver.JobManagerActor$$anonfun$spark$jobserver$JobManagerActor$$getJobFuture$4.apply(JobManagerActor.scala:244)", "scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)", "scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)", "java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)", "java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)", "java.lang.Thread.run(Thread.java:745)"] }
public class SimpleJob implements SnappySQLJob {
/**
*
*/
public SimpleJob() {
System.out.println(getClass().getSimpleName()+" Created");
}
@Override
public Object runJob(Object sparkContext, Config jobConfig) {
SnappyContext snappyContext = (SnappyContext)sparkContext;
System.out.println(getClass().getSimpleName()+".runJob: executed");
return null;
}
@Override
public SparkJobValidation validate(Object sparkContext, Config jobConfig) {
SnappyContext snappyContext = (SnappyContext)sparkContext;
System.out.println(getClass().getSimpleName()+".validate: executed");
return null;
}
}
16/08/05 17:44:07.352 UTC SnappyLeadJobServer-akka.actor.default-dispatcher-14 INFO JarManager:为app SimpleJobApp存储jar,1052字节16/08/05 17:44:07.368 UTC SnappyLeadJobServer-akka .actor.default-dispatcher-19 INFO LocalContextSupervisorActor:创建一个名为snappyContext1470419047337607598的SparkContext 16/08/05 17:44:07.369 UTC SnappyLeadJobServer-akka.actor.default-dispatcher-16 INFO JobManagerActor:启动actor spark.jobserver.JobManagerActor 16 / 08/05 17:44:07.371 UTC SnappyLeadJobServer-akka.actor.default-dispatcher-16 INFO JobStatusActor:开始演员spark.jobserver.JobStatusActor 16/08/05 17:44:07.371 UTC SnappyLeadJobServer-akka.actor.default-dispatcher -14 INFO JobResultActor:起始actor spark.jobserver.JobResultActor 16/08/05 17:44:07.371 UTC SnappyLeadJobServer-akka.actor.default-dispatcher-17 INFO LocalContextSupervisorActor:SparkContext snappyContext1470419047337607598 initialized 16/08/05 17:44:07.375 UTC SnappyLeadJobServer-akka.actor.default-disp atcher-16 INFO RddManagerActor:启动actor spark.jobserver.RddManagerActor 16/08/05 17:44:07.389 UTC SnappyLeadJobServer-akka.actor.default-dispatcher-17 INFO JobManagerActor:加载类snappydata.jobs.SimpleJob for app SimpleJobApp 16 / 08/05 17:44:07.389 UTC SnappyLeadJobServer-akka.actor.default-dispatcher-17 INFO SparkContext:在{{添加了JAR /tmp/spark-jobserver/filedao/data/SimpleJobApp-2016-08-05T17_44_07.353Z.jar 3}}时间戳1470419047389 16/08/05 17:44:07.390 UTC SnappyLeadJobServer-akka.actor.default-dispatcher-17 INFO ContextURLClassLoader:添加了URL文件:/ tmp / spark-jobserver / filedao / data / SimpleJobApp-2016- 08-05T17_44_07.353Z.jar到ContextURLClassLoader 16/08/05 17:44:07.390 UTC SnappyLeadJobServer-akka.actor.default-dispatcher-17 INFO JarUtils $:使用loader spark.jobserver.util加载对象snappydata.jobs.SimpleJob $ .ContextURLClassLoader @ 709f3e69 16/08/05 17:44:07.391 UTC SnappyLeadJobServer-akka.actor.default-dispatcher-17 INFO JarUtils $:加载类sna ppydata.jobs.SimpleJob使用loader spark.jobserver.util.ContextURLClassLoader@709f3e69 16/08/05 17:44:07.392 UTC SnappyLeadJobServer-akka.actor.default-dispatcher-17 INFO JobManagerActor:启动Spark工作376c8d23-6b49-4138- aadd-e4cff8f9f945 [snappydata.jobs.SimpleJob] ... 16/08/05 17:44:07.398 UTC pool-29-thread-1 INFO JobManagerActor:开始工作未来主题16/08/05 17:44:07.402 UTC SnappyLeadJobServer -akka.actor.default-dispatcher-17 INFO JobStatusActor:作业376c8d23-6b49-4138-aadd-e4cff8f9f945完成错误16/08/05 17:44:07.402 UTC pool-29-thread-2 WARN JobManagerActor:异常来自作业376c8d23-6b49-4138-aadd-e4cff8f9f945:scala.MatchError:null at spark.jobserver.JobManagerActor $$ anonfun $ spark $ jobserver $ JobManagerActor $$ getJobFuture $ 4.apply(JobManagerActor.scala:244)at scala.concurrent.impl .Future $ PromiseCompletingRunnable.liftedTree1 $ 1(Future.scala:24)at scala.concurrent.impl.Future $ PromiseCompletingRunnable.run(Future.scala:24)at java.util.concu rrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)at java.util.concurrent.ThreadPoolExecutor $ Worker.run(ThreadPoolExecutor.java:617)at java.lang.Thread.run(Thread.java:745)
答案 0 :(得分:2)
使用scala API的Java程序会出现此错误。我已使用Java API编写了SimpleJob。
public class SimpleJob extends JavaSnappySQLJob {
public SimpleJob() {
System.out.println(getClass().getSimpleName()+" Created");
}
@Override
public Object runJavaJob(SnappyContext snappyContext, Config config) {
System.out.println(getClass().getSimpleName()+".runJob:
executed");
return null;
}
@Override
public JSparkJobValidation isValidJob(SnappyContext snappyContext, Config config) {
System.out.println(getClass().getSimpleName()+".validate: executed");
return new JSparkJobValid();
}
}