SparkLauncher正在与其他用户一起启动Spark作业

时间:2019-06-18 17:40:47

标签: apache-spark oozie spark-launcher

我正在使用oozie工作流程,其中我正在使用SparkLauncher来与特定用户触发火花作业,因此我将用户设置为启动器配置,如下所示。

Map<String, String> envParams = new HashMap<>();
envParams.put("HADOOP_USER_NAME", "child_worker");
SparkLauncher launcher = new SparkLauncher(envParams);

但是 child_worker 用户无法启动spark作业。我正在访问一个相关的异常,该异常正在运行Oozie作业的用户无法访问child_worker目录。

Application application_1538580260012_381055 failed 2 times due to AM Container for appattempt_1538580260012_381055_000002 exited with exitCode: -1000
For more detailed output, check application tracking page:http://namenode1:8088/proxy/application_1538580260012_381055/Then, click on links to logs of each attempt.
Diagnostics: Permission denied: user=parent_user, access=EXECUTE, inode="/user/child_worker/.sparkStaging/application_1538580260012_381055":child_worker:hadoop:drwx------

0 个答案:

没有答案