Question

在新安装的Hadoop HDP 2.2.2.4上运行Oozie java操作时，例如尝试访问hdfs时，它会访问错误的文件系统： java.lang.IllegalArgumentException: Wrong FS: hdfs:/tmp/text.txt, expected: file:///

可以通过在Oozie操作中包含core-site.xml来修复它：

<file>hdfs:/path-to-core-site.xml-on-hdfs</file>

但是原因是什么，适当的解决办法是什么？

Answer 1

core-site.xml未包含在java-action的类路径中的原因是因为属性 mapreduce.application.classpath 指向错误的目录：

<snip>/etc/hadoop/conf/secure

应该指向

<snip>/etc/hadoop/conf

即，完整属性应该类似于 mapred-site.xml ：

<property>
  <name>mapreduce.application.classpath</name>
  <value>$PWD/mr-framework/hadoop/share/hadoop/mapreduce/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/*:$PWD/mr-framework/hadoop/share/hadoop/common/*:$PWD/mr-framework/hadoop/share/hadoop/common/lib/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/*:/usr/hdp/${hdp.version}/hadoop/lib/hadoop-lzo-0.6.0.${hdp.version}.jar:/etc/hadoop/conf</value>
</property>

Answer 2

这些文件包含在hadoop classpath中，据我所知，自HDP 2.2以来，你需要添加

// loading action conf prepared by Oozie
Configuration actionConf = new Configuration(false);
actionConf.addResource(new Path("file:///", System.getProperty("oozie.action.conf.xml")));

使用* -site.xml，您可以在oozie文档中获取详细信息 https://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html#a3.2.7_Java_Action

Oozie java-action不包含core-site.xml

2 个答案: