Oozie - 不识别nameNode的“maprfs:///”

时间:2015-10-19 05:26:45

标签: hadoop oozie mapr

我在自定义MapR沙箱中运行Oozie。以下是我的job.properties文件信息:

nameNode=maprfs:///
jobTracker=maprfs:///
queueName=default
examplesRoot=examples

oozie.wf.application.path=/user/user01/lab2/exercise4/zayd_hammoudeh_lab2_exercise4/workflow.xml

供参考,以下是我的workflow.xml文件:

    

<action name="stdev-calc">
    <map-reduce>
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>"${nameNode}"</name-node>
        <prepare>
            <delete path="/user/user01/lab2/exercise4/OUT"/>
        </prepare>
        <configuration>
            <property>
                <name>mapred.mapper.class</name>
                <value>Stdev.StdevMapper</value>
            </property>
            <property>
                <name>mapred.reducer.class</name>
                <value>Stdev.StdevReducer</value>
            </property>
            <property>
                <name>mapred.input.dir</name>
                <value>/user/user01/lab2/exercise4/DATA</value>
            </property>
            <property>
                <name>mapred.output.dir</name>
                <value>/user/user01/lab2/exercise4/OUT</value>
            </property>
        </configuration>
    </map-reduce>

    <ok to="end"/>
    <error to="fail"/>
</action>

<kill name="fail">
    <message>Exercise 4 failed with error message: [${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>

<end name="end"/>

当我尝试完成工作时,我在工作信息中得到以下信息:

Job ID : 0000004-151018210741238-oozie-mapr-W
------------------------------------------------------------------------------------------------------------------------------------
Workflow Name : aggregator-wf
App Path      : /user/user01/lab2/exercise4/zayd_hammoudeh_lab2_exercise4/workflow.xml
Status        : KILLED
Run           : 0
User          : user01
Group         : -
Created       : 2015-10-19 04:42 GMT
Started       : 2015-10-19 04:42 GMT
Last Modified : 2015-10-19 04:42 GMT
Ended         : 2015-10-19 04:42 GMT
CoordAction ID: -

Actions
------------------------------------------------------------------------------------------------------------------------------------
ID                                                                            Status    Ext ID                 Ext Status Err Code
------------------------------------------------------------------------------------------------------------------------------------
0000004-151018210741238-oozie-mapr-W@:start:                                  OK        -                      OK         -
------------------------------------------------------------------------------------------------------------------------------------
0000004-151018210741238-oozie-mapr-W@stdev-calc                               ERROR     -                      ERROR      IllegalArgumentException
------------------------------------------------------------------------------------------------------------------------------------
0000004-151018210741238-oozie-mapr-W@fail                                     OK        -                      OK         E0729
------------------------------------------------------------------------------------------------------------------------------------

我的日志显示错误为:“ [IllegalArgumentException:索引0处方案名称中的非法字符:”maprfs:///“] ”以下是我的日志:

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/mapr/oozie/oozie-4.1.0/lib/slf4j-simple-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/mapr/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.SimpleLoggerFactory]
2015-10-18 21:42:06,368  INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@:start:] Start action [0000004-151018210741238-oozie-mapr-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-10-18 21:42:06,369  INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@:start:] [***0000004-151018210741238-oozie-mapr-W@:start:***]Action status=DONE
2015-10-18 21:42:06,372  INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@:start:] [***0000004-151018210741238-oozie-mapr-W@:start:***]Action updated in DB!
2015-10-18 21:42:06,413  INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@stdev-calc] Start action [0000004-151018210741238-oozie-mapr-W@stdev-calc] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-10-18 21:42:06,468  WARN ActionStartXCommand:544 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@stdev-calc] Error starting action [stdev-calc]. ErrorType [ERROR], ErrorCode [IllegalArgumentException], Message [IllegalArgumentException: Illegal character in scheme name at index 0: "maprfs:///"]
org.apache.oozie.action.ActionExecutorException: IllegalArgumentException: Illegal character in scheme name at index 0: "maprfs:///"
        at org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:401)
        at org.apache.oozie.action.hadoop.JavaActionExecutor.addToCache(JavaActionExecutor.java:532)
        at org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:666)
        at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:884)
        at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
        at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
        at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
        at org.apache.oozie.command.XCommand.call(XCommand.java:281)
        at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
        at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
        at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Illegal character in scheme name at index 0: "maprfs:///"
        at java.net.URI.create(URI.java:859)
        at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:177)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:392)
        at org.apache.hadoop.fs.Path.getFileSystem(Path.java:310)
        at org.apache.oozie.util.JobUtils.addFileToClassPath(JobUtils.java:151)
        at org.apache.oozie.service.HadoopAccessorService$3.run(HadoopAccessorService.java:539)
        at org.apache.oozie.service.HadoopAccessorService$3.run(HadoopAccessorService.java:536)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566)
        at org.apache.oozie.service.HadoopAccessorService.addFileToClassPath(HadoopAccessorService.java:536)
        at org.apache.oozie.action.hadoop.JavaActionExecutor.addToCache(JavaActionExecutor.java:512)
        ... 12 more
Caused by: java.net.URISyntaxException: Illegal character in scheme name at index 0: "maprfs:///"
        at java.net.URI$Parser.fail(URI.java:2829)
        at java.net.URI$Parser.checkChars(URI.java:3002)
        at java.net.URI$Parser.checkChar(URI.java:3012)
        at java.net.URI$Parser.parse(URI.java:3028)
        at java.net.URI.<init>(URI.java:595)
        at java.net.URI.create(URI.java:857)
        ... 23 more
2015-10-18 21:42:06,468  WARN ActionStartXCommand:544 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@stdev-calc] Setting Action Status to [DONE]
2015-10-18 21:42:06,494  INFO ActionEndXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@stdev-calc] ERROR is considered as FAILED for SLA
2015-10-18 21:42:06,518  INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@fail] Start action [0000004-151018210741238-oozie-mapr-W@fail] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-10-18 21:42:06,518  INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@fail] [***0000004-151018210741238-oozie-mapr-W@fail***]Action status=DONE
2015-10-18 21:42:06,518  INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@fail] [***0000004-151018210741238-oozie-mapr-W@fail***]Action updated in DB!

我尽我所能找出问题的根源,但没有成功。对于namenode和jobtracker使用“maprfs:///”建议与mapr-oozie包捆绑在一起的示例。任何帮助都会非常感激。

1 个答案:

答案 0 :(得分:1)

我之前错过的问题是,当我在名称节点上收到错误时,由于&#34; job.properties&#34;的内容,它。文件。相反,它来自&#34; workflow.xml&#34;文件。我写错了:

<name-node>"${nameNode}"</name-node>

Oozie不会处理引号,如果删除它们,问题就会消失。最后的关键教训是,当您看到不容易解释的Oozie错误时,要小心检查workflow.xml和job.properties文件。