我尝试在集群模式下运行 SparkPi，但是当涉及到 reduce 时，执行程序似乎无法接收任何资源。
我这样提交了我的任务：
SELECT id,
CASE GREATEST(cars_cat1, cars_cat2, cars_cat_3)
WHEN cars_cat1 THEN 'cars_cat1'
WHEN cars_cat2 THEN 'cars_cat2'
WHEN cars_cat3 THEN 'cars_cat3'
END max_category
FROM tablename
我检查了 spark-kubernetes-driver 中的日志：
./bin/spark-submit \
--master k8s://172.16.2.120:6443 \
--deploy-mode cluster \
--name spark-pi \
--class org.apache.spark.examples.SparkPi \
--conf spark.executor.instances=5 \
--conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \
--conf spark.kubernetes.container.image.pullPolicy=Never \
--conf spark.kubernetes.container.image=ethanzhang1997/spark:test \
local:///opt/spark/examples/jars/spark-examples_2.12-3.1.1.jar
日志重复如上，然后我查看spark executor pods中的错误日志：
32787 [main] INFO org.apache.spark.scheduler.cluster.k8s.KubernetesClusterSchedulerBackend - SchedulerBackend is ready for scheduling beginning after waiting maxRegisteredResourcesWaitingTime: 30000000000(ns)
33285 [main] INFO org.apache.spark.SparkContext - Starting job: reduce at SparkPi.scala:38
33307 [dag-scheduler-event-loop] INFO org.apache.spark.scheduler.DAGScheduler - Got job 0 (reduce at SparkPi.scala:38) with 2 output partitions
33308 [dag-scheduler-event-loop] INFO org.apache.spark.scheduler.DAGScheduler - Final stage: ResultStage 0 (reduce at SparkPi.scala:38)
33308 [dag-scheduler-event-loop] INFO org.apache.spark.scheduler.DAGScheduler - Parents of final stage: List()
33310 [dag-scheduler-event-loop] INFO org.apache.spark.scheduler.DAGScheduler - Missing parents: List()
33316 [dag-scheduler-event-loop] INFO org.apache.spark.scheduler.DAGScheduler - Submitting ResultStage 0 (MapPartitionsRDD[1] at map at SparkPi.scala:34), which has no missing parents
33417 [dag-scheduler-event-loop] INFO org.apache.spark.storage.memory.MemoryStore - Block broadcast_0 stored as values in memory (estimated size 3.1 KiB, free 413.9 MiB)
33467 [dag-scheduler-event-loop] INFO org.apache.spark.storage.memory.MemoryStore - Block broadcast_0_piece0 stored as bytes in memory (estimated size 1816.0 B, free 413.9 MiB)
33471 [dispatcher-BlockManagerMaster] INFO org.apache.spark.storage.BlockManagerInfo - Added broadcast_0_piece0 in memory on spark-pi-8670aa7a611ba783-driver-svc.default.svc:7079 (size: 1816.0 B, free: 413.9 MiB)
33474 [dag-scheduler-event-loop] INFO org.apache.spark.SparkContext - Created broadcast 0 from broadcast at DAGScheduler.scala:1383
33492 [dag-scheduler-event-loop] INFO org.apache.spark.scheduler.DAGScheduler - Submitting 2 missing tasks from ResultStage 0 (MapPartitionsRDD[1] at map at SparkPi.scala:34) (first 15 tasks are for partitions Vector(0, 1))
33494 [dag-scheduler-event-loop] INFO org.apache.spark.scheduler.TaskSchedulerImpl - Adding task set 0.0 with 2 tasks resource profile 0
34386 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator - Going to request 1 executors from Kubernetes for ResourceProfile Id: 0, target: 5 running: 4.
34391 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.submit.KubernetesClientUtils - Spark configuration files loaded from Some(/opt/spark/conf) : workers,spark-env.sh
34393 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.features.BasicExecutorFeatureStep - Decommissioning not enabled, skipping shutdown script
36448 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator - Going to request 2 executors from Kubernetes for ResourceProfile Id: 0, target: 5 running: 3.
36451 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.submit.KubernetesClientUtils - Spark configuration files loaded from Some(/opt/spark/conf) : workers,spark-env.sh
36453 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.features.BasicExecutorFeatureStep - Decommissioning not enabled, skipping shutdown script
36541 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.submit.KubernetesClientUtils - Spark configuration files loaded from Some(/opt/spark/conf) : workers,spark-env.sh
36543 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.features.BasicExecutorFeatureStep - Decommissioning not enabled, skipping shutdown script
37594 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator - Going to request 1 executors from Kubernetes for ResourceProfile Id: 0, target: 5 running: 4.
37596 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.submit.KubernetesClientUtils - Spark configuration files loaded from Some(/opt/spark/conf) : workers,spark-env.sh
37597 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.features.BasicExecutorFeatureStep - Decommissioning not enabled, skipping shutdown script
40680 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator - Going to request 1 executors from Kubernetes for ResourceProfile Id: 0, target: 5 running: 4.
40685 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.submit.KubernetesClientUtils - Spark configuration files loaded from Some(/opt/spark/conf) : workers,spark-env.sh
40687 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.features.BasicExecutorFeatureStep - Decommissioning not enabled, skipping shutdown script
44748 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator - Going to request 1 executors from Kubernetes for ResourceProfile Id: 0, target: 5 running: 4.
44759 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.submit.KubernetesClientUtils - Spark configuration files loaded from Some(/opt/spark/conf) : workers,spark-env.sh
44762 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.features.BasicExecutorFeatureStep - Decommissioning not enabled, skipping shutdown script
46844 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator - Going to request 1 executors from Kubernetes for ResourceProfile Id: 0, target: 5 running: 4.
46847 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.submit.KubernetesClientUtils - Spark configuration files loaded from Some(/opt/spark/conf) : workers,spark-env.sh
46849 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.features.BasicExecutorFeatureStep - Decommissioning not enabled, skipping shutdown script
48525 [Timer-0] WARN org.apache.spark.scheduler.TaskSchedulerImpl - Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
48915 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator - Going to request 2 executors from Kubernetes for ResourceProfile Id: 0, target: 5 running: 3.
48921 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.submit.KubernetesClientUtils - Spark configuration files loaded from Some(/opt/spark/conf) : workers,spark-env.sh
48923 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.features.BasicExecutorFeatureStep - Decommissioning not enabled, skipping shutdown script
48995 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.submit.KubernetesClientUtils - Spark configuration files loaded from Some(/opt/spark/conf) : workers,spark-env.sh
48998 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.features.BasicExecutorFeatureStep - Decommissioning not enabled, skipping shutdown script
53067 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator - Going to request 1 executors from Kubernetes for ResourceProfile Id: 0, target: 5 running: 4.
53072 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.submit.KubernetesClientUtils - Spark configuration files loaded from Some(/opt/spark/conf) : workers,spark-env.sh
53075 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.features.BasicExecutorFeatureStep - Decommissioning not enabled, skipping shutdown script
56125 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator - Going to request 1 executors from Kubernetes for ResourceProfile Id: 0, target: 5 running: 4.
56128 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.submit.KubernetesClientUtils - Spark configuration files loaded from Some(/opt/spark/conf) : workers,spark-env.sh
56130 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.features.BasicExecutorFeatureStep - Decommissioning not enabled, skipping shutdown script
57210 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator - Going to request 2 executors from Kubernetes for ResourceProfile Id: 0, target: 5 running: 3.
57213 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.submit.KubernetesClientUtils - Spark configuration files loaded from Some(/opt/spark/conf) : workers,spark-env.sh
57214 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.features.BasicExecutorFeatureStep - Decommissioning not enabled, skipping shutdown script
57272 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.submit.KubernetesClientUtils - Spark configuration files loaded from Some(/opt/spark/conf) : workers,spark-env.sh
57273 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.features.BasicExecutorFeatureStep - Decommissioning not enabled, skipping shutdown script
60327 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator - Going to request 1 executors from Kubernetes for ResourceProfile Id: 0, target: 5 running: 4.
60329 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.submit.KubernetesClientUtils - Spark configuration files loaded from Some(/opt/spark/conf) : workers,spark-env.sh
60331 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.features.BasicExecutorFeatureStep - Decommissioning not enabled, skipping shutdown script
63519 [Timer-0] WARN org.apache.spark.scheduler.TaskSchedulerImpl - Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
64390 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator - Going to request 2 executors from Kubernetes for ResourceProfile Id: 0, target: 5 running: 3.
64394 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.submit.KubernetesClientUtils - Spark configuration files loaded from Some(/opt/spark/conf) : workers,spark-env.sh
64395 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.features.BasicExecutorFeatureStep - Decommissioning not enabled, skipping shutdown script
64469 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.submit.KubernetesClientUtils - Spark configuration files loaded from Some(/opt/spark/conf) : workers,spark-env.sh
64471 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.features.BasicExecutorFeatureStep - Decommissioning not enabled, skipping shutdown script
66525 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator - Going to request 1 executors from Kubernetes for ResourceProfile Id: 0, target: 5 running: 4.
66531 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.submit.KubernetesClientUtils - Spark configuration files loaded from Some(/opt/spark/conf) : workers,spark-env.sh
66533 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.features.BasicExecutorFeatureStep - Decommissioning not enabled, skipping shutdown script
68592 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator - Going to request 1 executors from Kubernetes for ResourceProfile Id: 0, target: 5 running: 4.
68594 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.submit.KubernetesClientUtils - Spark configuration files loaded from Some(/opt/spark/conf) : workers,spark-env.sh
68595 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.features.BasicExecutorFeatureStep - Decommissioning not enabled, skipping shutdown script
71646 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator - Going to request 1 executors from Kubernetes for ResourceProfile Id: 0, target: 5 running: 4.
71649 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.submit.KubernetesClientUtils - Spark configuration files loaded from Some(/opt/spark/conf) : workers,spark-env.sh
71650 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.features.BasicExecutorFeatureStep - Decommissioning not enabled, skipping shutdown script
76720 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator - Going to request 1 executors from Kubernetes for ResourceProfile Id: 0, target: 5 running: 4.
76723 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.submit.KubernetesClientUtils - Spark configuration files loaded from Some(/opt/spark/conf) : workers,spark-env.sh
76724 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.features.BasicExecutorFeatureStep - Decommissioning not enabled, skipping shutdown script
77782 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator - Going to request 1 executors from Kubernetes for ResourceProfile Id: 0, target: 5 running: 4.
77785 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.submit.KubernetesClientUtils - Spark configuration files loaded from Some(/opt/spark/conf) : workers,spark-env.sh
77787 [kubernetes-executor-snapshots-subscribers-1] INFO org.apache.spark.deploy.k8s.features.BasicExecutorFeatureStep - Decommissioning not enabled, skipping shutdown script
78520 [Timer-0] WARN org.apache.spark.scheduler.TaskSchedulerImpl - Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
我该如何解决这个问题？