oozie hive hadoop工作

时间:2013-03-21 06:55:37

标签: hadoop mapreduce hive sqoop oozie

嗨,我能够在蜂巢终端中运行一个配置单元命令但是当我尝试通过oozie运行它时我无法运行它获取一些错误和日志文件如下所示

2013-03-21 11:55:48,973  INFO ActionStartXCommand:539 - USER[biadmin] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000005-130314190356094-oozie-biad-W] ACTION[0000005-130314190356094-oozie-biad-W@hive-node] Start action [0000005-130314190356094-oozie-biad-W@hive-node] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2013-03-21 11:55:57,940  WARN HiveActionExecutor:542 - USER[biadmin] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000005-130314190356094-oozie-biad-W] ACTION[0000005-130314190356094-oozie-biad-W@hive-node] credentials is null for the action
2013-03-21 11:55:57,940  WARN HiveActionExecutor:542 - USER[biadmin] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000005-130314190356094-oozie-biad-W] ACTION[0000005-130314190356094-oozie-biad-W@hive-node] Could not find credentials properties for: null
2013-03-21 11:56:04,940  INFO HiveActionExecutor:539 - USER[biadmin] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000005-130314190356094-oozie-biad-W] ACTION[0000005-130314190356094-oozie-biad-W@hive-node] checking action, external ID [job_201303141654_0020] status [RUNNING]
2013-03-21 11:56:04,955  WARN ActionStartXCommand:542 - USER[biadmin] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000005-130314190356094-oozie-biad-W] ACTION[0000005-130314190356094-oozie-biad-W@hive-node] [***0000005-130314190356094-oozie-biad-W@hive-node***]Action status=RUNNING
2013-03-21 11:56:05,334  WARN ActionStartXCommand:542 - USER[biadmin] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000005-130314190356094-oozie-biad-W] ACTION[0000005-130314190356094-oozie-biad-W@hive-node] [***0000005-130314190356094-oozie-biad-W@hive-node***]Action updated in DB!
2013-03-21 11:56:33,520  INFO CallbackServlet:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000005-130314190356094-oozie-biad-W] ACTION[0000005-130314190356094-oozie-biad-W@hive-node] callback for action [0000005-130314190356094-oozie-biad-W@hive-node]
2013-03-21 11:56:34,214  INFO HiveActionExecutor:539 - USER[biadmin] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000005-130314190356094-oozie-biad-W] ACTION[0000005-130314190356094-oozie-biad-W@hive-node] action completed, external ID [job_201303141654_0020]
2013-03-21 11:56:34,365  WARN HiveActionExecutor:542 - USER[biadmin] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000005-130314190356094-oozie-biad-W] ACTION[0000005-130314190356094-oozie-biad-W@hive-node] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [10]
2013-03-21 11:56:35,059  INFO ActionEndXCommand:539 - USER[biadmin] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000005-130314190356094-oozie-biad-W] ACTION[0000005-130314190356094-oozie-biad-W@hive-node] ERROR is considered as FAILED for SLA
2013-03-21 11:56:35,364  WARN CoordActionUpdateXCommand:542 - USER[biadmin] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000005-130314190356094-oozie-biad-W] ACTION[-] E1100: Command precondition does not hold before execution, [, coord action is null], Error Code: E1100

3 个答案:

答案 0 :(得分:1)

这看起来像Oozie作业日志,它通常无法提供足够的信息来找到问题的根源。您应该查看map reduce作业的任务日志。找到它的最简单方法是转到Oozie Web控制台(即http://example.example.com:11000/oozie/),双击有问题的oozie作业打开它的作业信息,在作业信息选项卡上双击失败的Hive操作。在操作选项卡中应该有一个控制台URL字段。打开该地址并单击链接以查找有关任务的更多信息(即使是成功的,因为oozie可以吞下配置单元故障并将成功报告给任务跟踪器。根据我的经验,有用的日志始终处于成功的映射任务中)。看看日志,希望,但不一定,你会找到一些东西,从“蜂巢失败”缩小它。

这不是问题的真实答案,所以如果有人想将此作为评论发布并将此答案标记为删除,请执行此操作。

答案 1 :(得分:0)

错误来自Oozie,协调员没有启动工作流程,即如果您启动Oozie与root用户停止服务并重新启动Oozie与您安装Oozie的用户。现在重新运行工作流程。

这可以解决你的问题!!

答案 2 :(得分:0)

以上错误是因为您使用了不匹配的hadoop和hive版本。要清除此错误,请使用所需版本重建oozie。 要使用requied版本重建oozie,请在OOZIE_HOME中编辑pom.xml文件。

<hadoop.version>x.x.x</hadoop.version>
 <hive.version>0.x.x</hive.version>
 <pig.version>0.x.x</pig.version> 

编辑pom文件后,运行OOZIE_HOME / bin / mkdistro.sh -DskipTests。