Spark版本1.3.0
在群集模式下向作业集群提交作业时出错
./spark-submit --class org.apache.spark.examples.streaming.JavaDirectKafkaWordCount --deploy-mode cluster wordcount-0.1.jar 172.20.5.174:9092,172.20.9.50:9092,172.20.7.135:9092 log
收率:
Spark assembly has been built with Hive, including Datanucleus jars on classpath
Running Spark using the REST application submission protocol.
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
15/04/14 16:41:10 INFO StandaloneRestClient: Submitting a request to launch an application in spark://172.20.9.151:7077.
Warning: Master endpoint spark://172.20.9.151:7077 was not a REST server. Falling back to legacy submission gateway instead.
15/04/14 16:41:11 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Sending launch command to spark://172.20.9.151:7077
Error connecting to master spark://172.20.9.151:7077 (akka.tcp://sparkMaster@172.20.9.151:7077), exiting.
答案 0 :(得分:20)
默认情况下,Master Spark REST URL位于端口6066上。 因此,您应该将此视为主终端:spark://172.20.9.151:6066。
如果您访问Spark Web控制台(http://master:8080),您将获得群集中各个端点的详细信息。