Question

我有一个Oozie java动作工作流程设置，我计划使用Oozie协调员来安排。 java动作运行一个Camus Job，它的jar和属性配置文件放在workflow / lib目录中。关于如何将-P参数传递给它的任何想法？目前，我正在做这样的事情：

<workflow-app xmlns="uri:oozie:workflow:0.5" name="camus-wf">
    <start to="camusJob"/>
    <action name="camusJob">
        <java>
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <configuration>
            <property>
                    <name>mapred.job.name</name>
                    <value>camusJob</value>
                </property>
                <property>
                    <name>mapred.job.queue.name</name>
                    <value>${queueName}</value>
                </property>
            </configuration>
            <main-class>com.linkedin.camus.etl.kafka.CamusJob</main-class>
    <arg>-P</arg>
    <arg>${camusJobProperties}</arg>
        </java>
        <ok to="end"/>
        <error to="fail"/>
    </action>
    <kill name="fail">
        <message>${wf:errorMessage(wf:lastErrorNode())}</message>
    </kill>
    <end name="end"/>
</workflow-app>

camusJobProperties的样子

hdfs://10.0.2.15:8020/coordCamusJob/workflowAppPath/lib/config.properties

但是工作流程似乎没有运行（陷入PREP）。任何想法如何解决这个问题？

谢谢！

编辑：在更正我的nameNode URL后，我可以看到我收到以下错误：

ACTION[0000002-150804091125207-oozie-oozi-W@camusJob] Launcher exception: java.lang.IllegalArgumentException: Wrong FS: hdfs://10.0.2.15:8020/user/root/app/workflow/lib/config.properties, expected: file:///
org.apache.oozie.action.hadoop.JavaMainException: java.lang.IllegalArgumentException: Wrong FS: hdfs://10.0.2.15:8020/user/root/app/workflow/lib/config.properties, expected: file:///
    at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:58)
    at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
    at org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:36)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.IllegalArgumentException: Wrong FS: hdfs://10.0.2.15:8020/user/root/app/workflow/lib/config.properties, expected: file:///
    at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:645)
    at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:82)
    at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:603)
    at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:821)
    at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:598)
    at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:414)
    at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:140)
    at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:341)
    at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:766)
    at com.linkedin.camus.etl.kafka.CamusJob.run(CamusJob.java:679)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
    at com.linkedin.camus.etl.kafka.CamusJob.main(CamusJob.java:646)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:55)
    ... 15 more

所以基本上我的问题是当属性文件位于HDFS中时（特别是在workflow / lib目录中），如何传递属性文件参数

Answer 1

对于问题的第一部分：可能是由于namenode或jobtracker

的网址不正确

对于第二部分：您必须配置 core-site.xml ，将属性fs.defaultFS配置为hdfs://host:port/

另外，在您的config对象

中的core-site.xml的Java程序集路径中

将属性文件传递给Oozie Java Action

1 个答案: