我已经在centos8上安装了具有以下版本mesos 1.9.0和spark 2.4.5的mesos / sparkcluster。 我试图开始跟随火花工作 / appl / spark / bin / spark-submit --verbose --deploy-mode cluster --executor-memory 1G --master mesos://mesos.master:7077 /appl/spark-shared/lib/pi.py 10
所有作业均以退出代码127退出。
sh:第0行:cd:参数过多 sh:bin / spark-submit:没有这样的文件或目录
是否可能不支持centos8? 我不知道了 非常感谢!
以下是错误日志:
I0326 20:57:30.080410 66713 fetcher.cpp:562] Fetcher Info: {"cache_directory":"/tmp/mesos/fetch/mesos","items":[{"action":"BYPASS_CACHE","uri":{"cache":false,"extract":true,"value":"/appl/spark-shared/lib/pi.py"}},{"action":"BYPASS_CACHE","uri":{"cache":false,"extract":true,"value":"file:///appl/spark-shared/lib/spark-2.4.5-bin-hadoop2.7.tgz"}}],"sandbox_directory":"/appl/work/mesos/slaves/2489fe38-4b0d-4794-b720-20628066b58d-S0/frameworks/44d55039-16e0-4db4-b9d2-bbc73af27ff5-0000/executors/driver-20200326205729-0006/runs/5f5da7bf-4825-497e-a4fc-41cc3e5f6fa0","stall_timeout":{"nanoseconds":60000000000},"user":"mesos"}
I0326 20:57:30.101250 66713 fetcher.cpp:459] Fetching URI '/appl/spark-shared/lib/pi.py'
I0326 20:57:30.101300 66713 fetcher.cpp:290] Fetching '/appl/spark-shared/lib/pi.py' directly into the sandbox directory
W0326 20:57:30.105692 66713 fetcher.cpp:332] Copying instead of extracting resource from URI with 'extract' flag, because it does not seem to be an archive: /appl/spark-shared/lib/pi.py
I0326 20:57:30.105773 66713 fetcher.cpp:618] Fetched '/appl/spark-shared/lib/pi.py' to '/appl/work/mesos/slaves/2489fe38-4b0d-4794-b720-20628066b58d-S0/frameworks/44d55039-16e0-4db4-b9d2-bbc73af27ff5-0000/executors/driver-20200326205729-0006/runs/5f5da7bf-4825-497e-a4fc-41cc3e5f6fa0/pi.py'
I0326 20:57:30.105785 66713 fetcher.cpp:459] Fetching URI 'file:///appl/spark-shared/lib/spark-2.4.5-bin-hadoop2.7.tgz'
I0326 20:57:30.105790 66713 fetcher.cpp:290] Fetching 'file:///appl/spark-shared/lib/spark-2.4.5-bin-hadoop2.7.tgz' directly into the sandbox directory
I0326 20:57:31.888286 66713 fetcher.cpp:618] Fetched 'file:///appl/spark-shared/lib/spark-2.4.5-bin-hadoop2.7.tgz' to '/appl/work/mesos/slaves/2489fe38-4b0d-4794-b720-20628066b58d-S0/frameworks/44d55039-16e0-4db4-b9d2-bbc73af27ff5-0000/executors/driver-20200326205729-0006/runs/5f5da7bf-4825-497e-a4fc-41cc3e5f6fa0/spark-2.4.5-bin-hadoop2.7.tgz'
I0326 20:57:31.888345 66713 fetcher.cpp:623] Successfully fetched all URIs into '/appl/work/mesos/slaves/2489fe38-4b0d-4794-b720-20628066b58d-S0/frameworks/44d55039-16e0-4db4-b9d2-bbc73af27ff5-0000/executors/driver-20200326205729-0006/runs/5f5da7bf-4825-497e-a4fc-41cc3e5f6fa0'
Overwriting environment variable 'MESOS_NATIVE_JAVA_LIBRARY'
I0326 20:57:32.194804 66769 exec.cpp:164] Version: 1.9.0
I0326 20:57:32.219038 66779 exec.cpp:237] Executor registered on agent 2489fe38-4b0d-4794-b720-20628066b58d-S0
I0326 20:57:32.224376 66776 executor.cpp:190] Received SUBSCRIBED event
I0326 20:57:32.225698 66776 executor.cpp:194] Subscribed executor on mesos.master
I0326 20:57:32.226032 66776 executor.cpp:190] Received LAUNCH event
I0326 20:57:32.228232 66776 executor.cpp:722] Starting task driver-20200326205729-0006
I0326 20:57:32.255175 66776 executor.cpp:738] Forked command at 66782
sh: line 0: cd: too many arguments
sh: bin/spark-submit: No such file or directory
I0326 20:57:32.500557 66773 executor.cpp:1039] Command exited with status 127 (pid: 66782)
I0326 20:57:33.521730 66781 process.cpp:935] Stopped the socket accept loop