我在自定义MapR沙箱中运行Oozie。以下是我的job.properties文件信息:
nameNode=maprfs:///
jobTracker=maprfs:///
queueName=default
examplesRoot=examples
oozie.wf.application.path=/user/user01/lab2/exercise4/zayd_hammoudeh_lab2_exercise4/workflow.xml
供参考,以下是我的workflow.xml文件:
<action name="stdev-calc">
<map-reduce>
<job-tracker>${jobTracker}</job-tracker>
<name-node>"${nameNode}"</name-node>
<prepare>
<delete path="/user/user01/lab2/exercise4/OUT"/>
</prepare>
<configuration>
<property>
<name>mapred.mapper.class</name>
<value>Stdev.StdevMapper</value>
</property>
<property>
<name>mapred.reducer.class</name>
<value>Stdev.StdevReducer</value>
</property>
<property>
<name>mapred.input.dir</name>
<value>/user/user01/lab2/exercise4/DATA</value>
</property>
<property>
<name>mapred.output.dir</name>
<value>/user/user01/lab2/exercise4/OUT</value>
</property>
</configuration>
</map-reduce>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Exercise 4 failed with error message: [${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
当我尝试完成工作时,我在工作信息中得到以下信息:
Job ID : 0000004-151018210741238-oozie-mapr-W
------------------------------------------------------------------------------------------------------------------------------------
Workflow Name : aggregator-wf
App Path : /user/user01/lab2/exercise4/zayd_hammoudeh_lab2_exercise4/workflow.xml
Status : KILLED
Run : 0
User : user01
Group : -
Created : 2015-10-19 04:42 GMT
Started : 2015-10-19 04:42 GMT
Last Modified : 2015-10-19 04:42 GMT
Ended : 2015-10-19 04:42 GMT
CoordAction ID: -
Actions
------------------------------------------------------------------------------------------------------------------------------------
ID Status Ext ID Ext Status Err Code
------------------------------------------------------------------------------------------------------------------------------------
0000004-151018210741238-oozie-mapr-W@:start: OK - OK -
------------------------------------------------------------------------------------------------------------------------------------
0000004-151018210741238-oozie-mapr-W@stdev-calc ERROR - ERROR IllegalArgumentException
------------------------------------------------------------------------------------------------------------------------------------
0000004-151018210741238-oozie-mapr-W@fail OK - OK E0729
------------------------------------------------------------------------------------------------------------------------------------
我的日志显示错误为:“ [IllegalArgumentException:索引0处方案名称中的非法字符:”maprfs:///“] ”以下是我的日志:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/mapr/oozie/oozie-4.1.0/lib/slf4j-simple-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/mapr/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.SimpleLoggerFactory]
2015-10-18 21:42:06,368 INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@:start:] Start action [0000004-151018210741238-oozie-mapr-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-10-18 21:42:06,369 INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@:start:] [***0000004-151018210741238-oozie-mapr-W@:start:***]Action status=DONE
2015-10-18 21:42:06,372 INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@:start:] [***0000004-151018210741238-oozie-mapr-W@:start:***]Action updated in DB!
2015-10-18 21:42:06,413 INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@stdev-calc] Start action [0000004-151018210741238-oozie-mapr-W@stdev-calc] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-10-18 21:42:06,468 WARN ActionStartXCommand:544 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@stdev-calc] Error starting action [stdev-calc]. ErrorType [ERROR], ErrorCode [IllegalArgumentException], Message [IllegalArgumentException: Illegal character in scheme name at index 0: "maprfs:///"]
org.apache.oozie.action.ActionExecutorException: IllegalArgumentException: Illegal character in scheme name at index 0: "maprfs:///"
at org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:401)
at org.apache.oozie.action.hadoop.JavaActionExecutor.addToCache(JavaActionExecutor.java:532)
at org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:666)
at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:884)
at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
at org.apache.oozie.command.XCommand.call(XCommand.java:281)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Illegal character in scheme name at index 0: "maprfs:///"
at java.net.URI.create(URI.java:859)
at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:177)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:392)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:310)
at org.apache.oozie.util.JobUtils.addFileToClassPath(JobUtils.java:151)
at org.apache.oozie.service.HadoopAccessorService$3.run(HadoopAccessorService.java:539)
at org.apache.oozie.service.HadoopAccessorService$3.run(HadoopAccessorService.java:536)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566)
at org.apache.oozie.service.HadoopAccessorService.addFileToClassPath(HadoopAccessorService.java:536)
at org.apache.oozie.action.hadoop.JavaActionExecutor.addToCache(JavaActionExecutor.java:512)
... 12 more
Caused by: java.net.URISyntaxException: Illegal character in scheme name at index 0: "maprfs:///"
at java.net.URI$Parser.fail(URI.java:2829)
at java.net.URI$Parser.checkChars(URI.java:3002)
at java.net.URI$Parser.checkChar(URI.java:3012)
at java.net.URI$Parser.parse(URI.java:3028)
at java.net.URI.<init>(URI.java:595)
at java.net.URI.create(URI.java:857)
... 23 more
2015-10-18 21:42:06,468 WARN ActionStartXCommand:544 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@stdev-calc] Setting Action Status to [DONE]
2015-10-18 21:42:06,494 INFO ActionEndXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@stdev-calc] ERROR is considered as FAILED for SLA
2015-10-18 21:42:06,518 INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@fail] Start action [0000004-151018210741238-oozie-mapr-W@fail] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-10-18 21:42:06,518 INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@fail] [***0000004-151018210741238-oozie-mapr-W@fail***]Action status=DONE
2015-10-18 21:42:06,518 INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@fail] [***0000004-151018210741238-oozie-mapr-W@fail***]Action updated in DB!
“
我尽我所能找出问题的根源,但没有成功。对于namenode和jobtracker使用“maprfs:///”建议与mapr-oozie包捆绑在一起的示例。任何帮助都会非常感激。
答案 0 :(得分:1)
我之前错过的问题是,当我在名称节点上收到错误时,由于&#34; job.properties&#34;的内容,它不。文件。相反,它来自&#34; workflow.xml&#34;文件。我写错了:
<name-node>"${nameNode}"</name-node>
Oozie不会处理引号,如果删除它们,问题就会消失。最后的关键教训是,当您看到不容易解释的Oozie错误时,要小心检查workflow.xml和job.properties文件。