Spark Launcher传递多个JVM选项

时间:2018-07-09 17:52:03

标签: spark-launcher

我正在尝试通过Spark启动器传递多个JVM选项,但是它抛出了无法识别的选项。有任何想法吗。我的代码是..

SparkAppHandle handle = new SparkLauncher()
                .setSparkHome("/opt/spark/dist")
                .setAppResource("http://130.251.135.21:6521/repository/test.jar")
                .setMainClass("com.Ravi.test.mainClass")
                .setMaster("mesos://spark-dispatcher.marathon.l4lb.thisdcos.directory:7077")
                .setDeployMode("cluster")
                .setConf("spark.mesos.executor.home","/opt/spark/dist")
                .setConf(SparkLauncher.EXECUTOR_EXTRA_JAVA_OPTIONS,"-XX:-ResizePLAB -XX:+ParallelRefProcEnabled")
                .setConf(SparkLauncher.DRIVER_MEMORY, "20g")
                .setConf(SparkLauncher.EXECUTOR_MEMORY, "300g")
                .setConf(SparkLauncher.EXECUTOR_CORES, "25")
                .setConf("spark.cores.max","100")
                .startApplication();

0 个答案:

没有答案