我一直在使用spark-submit以群集模式启动应用程序,通常是这样的:
spark-submit ^
--master spark://master_ip_from_web_ui:6066 ^
--class MyClass ^
--deploy-mode cluster ^
--conf "maybesomeconf=val"
path_to_app_jar_on_worker_nodes ^
arg1 args
主Web ui已启动并正在运行,并显示连接到群集的所有工作人员。但是,我在提交申请后得到了这个回复:
16/07/06 14:47:02 INFO RestSubmissionClient: Submitting a request to launch an application in spark://master_ip:6066.
16/07/06 14:47:03 WARN RestSubmissionClient: Unable to connect to server spark://master_ip:6066.
Warning: Master endpoint spark://master_ip:6066 was not a REST server. Falling back to legacy submission gateway instead.