spark提交应用程序报告“ DriverRunner:命令退出,状态为1,之后重新启动”

时间:2019-04-21 13:20:06

标签: apache-spark

如果提交了如下所示的内置示例,则我在独立集群中设置了火花。

  ./bin/spark-submit \
  --class JavaWordCount \
  --master spark://192.168.77.130:7077 \
  /root/spark/examples/jars/spark-examples_2.11-2.4.1.jar \
  /tmp/abc.txt

然后,工作节点中的日志如下所示。

19/04/21 09:07:54 INFO DriverRunner: Launch Command: "/root/jdk1.8.0_191/bin/java" "-cp" "/root/spark/conf/:/root/spark/jars/*" "-Xmx1024M" "-Dspark.submit.deployMode=cluster" "-Dspark.master=spark://192.168.77.130:7077" "-Dspark.app.name=JavaWordCount" "-Dspark.jars=file:/root/spark/examples/jars/spark-examples_2.11-2.4.1.jar" "-Dspark.driver.supervise=true" "-Dspark.rpc.askTimeout=10s" "org.apache.spark.deploy.worker.DriverWrapper" "spark://Worker@192.168.77.131:34646" "/root/spark/work/driver-20190421090754-0000/spark-examples_2.11-2.4.1.jar" "JavaWordCount" "/tmp/abc.txt"
19/04/21 09:07:57 INFO DriverRunner: Command exited with status 1, re-launching after 1 s.
19/04/21 09:07:58 INFO DriverRunner: Launch Command: "/root/jdk1.8.0_191/bin/java" "-cp" "/root/spark/conf/:/root/spark/jars/*" "-Xmx1024M" "-Dspark.submit.deployMode=cluster" "-Dspark.master=spark://192.168.77.130:7077" "-Dspark.app.name=JavaWordCount" "-Dspark.jars=file:/root/spark/examples/jars/spark-examples_2.11-2.4.1.jar" "-Dspark.driver.supervise=true" "-Dspark.rpc.askTimeout=10s" "org.apache.spark.deploy.worker.DriverWrapper" "spark://Worker@192.168.77.131:34646" "/root/spark/work/driver-20190421090754-0000/spark-examples_2.11-2.4.1.jar" "JavaWordCount" "/tmp/abc.txt"
19/04/21 09:08:00 INFO DriverRunner: Command exited with status 1, re-launching after 2 s.
19/04/21 09:08:02 INFO DriverRunner: Launch Command: "/root/jdk1.8.0_191/bin/java" "-cp" "/root/spark/conf/:/root/spark/jars/*" "-Xmx1024M" "-Dspark.submit.deployMode=cluster" "-Dspark.master=spark://192.168.77.130:7077" "-Dspark.app.name=JavaWordCount" "-Dspark.jars=file:/root/spark/examples/jars/spark-examples_2.11-2.4.1.jar" "-Dspark.driver.supervise=true" "-Dspark.rpc.askTimeout=10s" "org.apache.spark.deploy.worker.DriverWrapper" "spark://Worker@192.168.77.131:34646" "/root/spark/work/driver-20190421090754-0000/spark-examples_2.11-2.4.1.jar" "JavaWordCount" "/tmp/abc.txt"
19/04/21 09:08:05 INFO DriverRunner: Command exited with status 1, re-launching after 4 s.
19/04/21 09:08:09 INFO DriverRunner: Launch Command: "/root/jdk1.8.0_191/bin/java" "-cp" "/root/spark/conf/:/root/spark/jars/*" "-Xmx1024M" "-Dspark.submit.deployMode=cluster" "-Dspark.master=spark://192.168.77.130:7077" "-Dspark.app.name=JavaWordCount" "-Dspark.jars=file:/root/spark/examples/jars/spark-examples_2.11-2.4.1.jar" "-Dspark.driver.supervise=true" "-Dspark.rpc.askTimeout=10s" "org.apache.spark.deploy.worker.DriverWrapper" "spark://Worker@192.168.77.131:34646" "/root/spark/work/driver-20190421090754-0000/spark-examples_2.11-2.4.1.jar" "JavaWordCount" "/tmp/abc.txt"
19/04/21 09:08:11 INFO DriverRunner: Command exited with status 1, re-launching after 8 s.
19/04/21 09:08:19 INFO DriverRunner: Launch Command: "/root/jdk1.8.0_191/bin/java" "-cp" "/root/spark/conf/:/root/spark/jars/*" "-Xmx1024M" "-Dspark.submit.deployMode=cluster" "-Dspark.master=spark://192.168.77.130:7077" "-Dspark.app.name=JavaWordCount" "-Dspark.jars=file:/root/spark/examples/jars/spark-examples_2.11-2.4.1.jar" "-Dspark.driver.supervise=true" "-Dspark.rpc.askTimeout=10s" "org.apache.spark.deploy.worker.DriverWrapper" "spark://Worker@192.168.77.131:34646" "/root/spark/work/driver-20190421090754-0000/spark-examples_2.11-2.4.1.jar" "JavaWordCount" "/tmp/abc.txt"
19/04/21 09:08:22 INFO DriverRunner: Command exited with status 1, re-launching after 16 s.
19/04/21 09:08:38 INFO DriverRunner: Launch Command: "/root/jdk1.8.0_191/bin/java" "-cp" "/root/spark/conf/:/root/spark/jars/*" "-Xmx1024M" "-Dspark.submit.deployMode=cluster" "-Dspark.master=spark://192.168.77.130:7077" "-Dspark.app.name=JavaWordCount" "-Dspark.jars=file:/root/spark/examples/jars/spark-examples_2.11-2.4.1.jar" "-Dspark.driver.supervise=true" "-Dspark.rpc.askTimeout=10s" "org.apache.spark.deploy.worker.DriverWrapper" "spark://Worker@192.168.77.131:34646" "/root/spark/work/driver-20190421090754-0000/spark-examples_2.11-2.4.1.jar" "JavaWordCount" "/tmp/abc.txt"
19/04/21 09:08:41 INFO DriverRunner: Command exited with status 1, re-launching after 32 s.

下面是master的仪表板。 enter image description here

0 个答案:

没有答案