关注the instructions here我一直在尝试使用远程网址向minikube提交火花作业:
minikube start
bin/spark-submit --master k8s://https://192.168.99.100:8443 --deploy-mode cluster --name spark-pi --class org.apache.spark.examples.SparkPi --conf spark.executor.instances=1 --conf spark.kubernetes.container.image=<default-spark-k8s-image-build> --conf spark.kubernetes.namespace=spark <https://remote-location-with-spark-example-jar>
pod失败,当我描述它时,我收到错误 configmaps&#34; spark-pi-ad386ea0f7e4333dbd2a0ad705e94d66-init-config&#34;找不到:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 50s default-scheduler Successfully assigned spark-pi-ad386ea0f7e4333dbd2a0ad705e94d66-driver to minikube
Warning FailedMount 49s kubelet, minikube MountVolume.SetUp failed for volume "spark-init-properties" : **configmaps "spark-pi-ad386ea0f7e4333dbd2a0ad705e94d66-init-config" not found**
Normal SuccessfulMountVolume 49s kubelet, minikube MountVolume.SetUp succeeded for volume "download-jars-volume"
Normal SuccessfulMountVolume 49s kubelet, minikube MountVolume.SetUp succeeded for volume "download-files-volume"
Normal SuccessfulMountVolume 49s kubelet, minikube MountVolume.SetUp succeeded for volume "default-token-4ghj8"
Normal SuccessfulMountVolume 49s kubelet, minikube MountVolume.SetUp succeeded for volume "spark-init-properties"
Normal Pulled 49s kubelet, minikube Container image "timg-spark/spark:latest" already present on machine
Normal Created 49s kubelet, minikube Created container
Normal Started 48s kubelet, minikube Started container
Normal Pulled 43s kubelet, minikube Container image "timmeh/spark:latest" already present on machine
Normal Created 43s kubelet, minikube Created container
Normal Started 43s kubelet, minikube Started container
然而,在创建任何配置映射的文档中没有提及,并且因为在运行spark-submit之前不知道configmap的名称,所以我无法提前创建一个以获取更多信息
现在我的计划是通过将jar文件烘焙到spark docker图像来解决,但是如果有人知道为什么这会失败那就太棒了!