oozie工作流shellAction

时间:2014-03-31 10:27:55

标签: shell hadoop hdfs oozie

我正在尝试使用包含以下命令的shell脚本在oozie工作流中运行shell操作:

.#!/bin/bash

hadoop fs -mkdir /user/mathon_k/shell_dir

.#echo "hello "

我收到以下错误:

Main class [org.apache.oozie.action.hadoop.ShellMain], exit code [1]

日志

  

2014-03-28 15:11:31,951 INFO   org.apache.oozie.command.wf.ActionStartXCommand:USER [mathon_k]   GROUP [ - ] TOKEN [] APP [oo_20140328]   JOB [0000028-140328083804680-Oozie的-oozi-W]   行动[0000028-140328083804680-oozie-oozi-W @:开始:]开始行动   具有用户重试状态的[0000028-140328083804680-oozie-oozi-W @:start:]:   userRetryCount [0],userRetryMax [0],userRetryInterval [10]   2014-03-28 15:11:31,952警告   org.apache.oozie.command.wf.ActionStartXCommand:USER [mathon_k]   GROUP [ - ] TOKEN [] APP [oo_20140328]   JOB [0000028-140328083804680-Oozie的-oozi-W]   ACTION [0000028-140328083804680-Oozie的-oozi-W @:开始:]   [ 0000028-140328083804680-oozie-oozi-W @:start: ]操作状态=完成   2014-03-28 15:11:31,952警告   org.apache.oozie.command.wf.ActionStartXCommand:USER [mathon_k]   GROUP [ - ] TOKEN [] APP [oo_20140328]   JOB [0000028-140328083804680-Oozie的-oozi-W]   ACTION [0000028-140328083804680-Oozie的-oozi-W @:开始:]   [ 0000028-140328083804680-oozie-oozi-W @:start: ]已更新   D B! 2014-03-28 15:11:32,027 INFO   org.apache.oozie.command.wf.ActionStartXCommand:USER [mathon_k]   GROUP [ - ] TOKEN [] APP [oo_20140328]   JOB [0000028-140328083804680-Oozie的-oozi-W]   行动[0000028-140328083804680-oozie-oozi-W @ oozie_20140328]开始   行动[0000028-140328083804680-oozie-oozi-W @ oozie_20140328]与   user-retry state:userRetryCount [0],userRetryMax [0],   userRetryInterval [10] 2014-03-28 15:11:32,170警告   org.apache.oozie.action.hadoop.ShellActionExecutor:USER [mathon_k]   GROUP [ - ] TOKEN [] APP [oo_20140328]   JOB [0000028-140328083804680-Oozie的-oozi-W]   ACTION [0000028-140328083804680-Oozie的-oozi-W @ oozie_20140328]   对于操作,凭据为空2014-03-28 15:11:32,669 INFO   org.apache.oozie.action.hadoop.ShellActionExecutor:USER [mathon_k]   GROUP [ - ] TOKEN [] APP [oo_20140328]   JOB [0000028-140328083804680-Oozie的-oozi-W]   行动[0000028-140328083804680-oozie-oozi-W @ oozie_20140328]检查   操作,外部ID [job_201403280835_0020]状态[运行]   2014-03-28 15:11:32,772警告   org.apache.oozie.command.wf.ActionStartXCommand:USER [mathon_k]   GROUP [ - ] TOKEN [] APP [oo_20140328]   JOB [0000028-140328083804680-Oozie的-oozi-W]   ACTION [0000028-140328083804680-Oozie的-oozi-W @ oozie_20140328]   [ * 0000028-140328083804680-Oozie的-oozi-W @ oozie_20140328 * ]操作   status = RUNNING 2014-03-28 15:11:32,772警告   org.apache.oozie.command.wf.ActionStartXCommand:USER [mathon_k]   GROUP [ - ] TOKEN [] APP [oo_20140328]   JOB [0000028-140328083804680-Oozie的-oozi-W]   ACTION [0000028-140328083804680-Oozie的-oozi-W @ oozie_20140328]   [ * 0000028-140328083804680-Oozie的-oozi-W @ oozie_20140328 * ]操作   在DB更新! 2014-03-28 15:11:45,254 INFO   org.apache.oozie.servlet.CallbackServlet:USER [ - ] GROUP [ - ] TOKEN [ - ]   APP [ - ] JOB [0000028-140328083804680-oozie-oozi-W]   行动[0000028-140328083804680-oozie-oozi-W @ oozie_20140328]回调   行动[0000028-140328083804680-oozie-oozi-W @ oozie_20140328]   2014-03-28 15:11:45,332 INFO   org.apache.oozie.action.hadoop.ShellActionExecutor:USER [mathon_k]   GROUP [ - ] TOKEN [] APP [oo_20140328]   JOB [0000028-140328083804680-Oozie的-oozi-W]   行动[0000028-140328083804680-oozie-oozi-W @ oozie_20140328]行动   已完成,外部身份证[job_201403280835_0020] 2014-03-28 15:11:45,358   警告org.apache.oozie.action.hadoop.ShellActionExecutor:   USER [mathon_k] GROUP [ - ] TOKEN [] APP [oo_20140328]   JOB [0000028-140328083804680-Oozie的-oozi-W]   行动[0000028-140328083804680-oozie-oozi-W @ oozie_20140328]发射器   错误,原因:主类[org.apache.oozie.action.hadoop.ShellMain],   退出代码[1] 2014-03-28 15:11:45,518 INFO   org.apache.oozie.command.wf.ActionEndXCommand:USER [mathon_k] GROUP [ - ]   TOKEN [] APP [oo_20140328] JOB [0000028-140328083804680-oozie-oozi-W]   行动[0000028-140328083804680-oozie-oozi-W @ oozie_20140328]错误是   被认为是失败的SLA 2014-03-28 15:11:45,563 INFO   org.apache.oozie.command.wf.ActionStartXCommand:USER [mathon_k]   GROUP [ - ] TOKEN [] APP [oo_20140328]   JOB [0000028-140328083804680-Oozie的-oozi-W]   行动[0000028-140328083804680-oozie-oozi-W @ kill]开始行动   具有用户重试状态的[0000028-140328083804680-oozie-oozi-W @ kill]:   userRetryCount [0],userRetryMax [0],userRetryInterval [10]   2014-03-28 15:11:45,563警告   org.apache.oozie.command.wf.ActionStartXCommand:USER [mathon_k]   GROUP [ - ] TOKEN [] APP [oo_20140328]   JOB [0000028-140328083804680-Oozie的-oozi-W]   ACTION [0000028-140328083804680-Oozie的-oozi-W @杀]   [ 0000028-140328083804680-oozie-oozi-W @ kill ]行动状态=已完成   2014-03-28 15:11:45,563警告   org.apache.oozie.command.wf.ActionStartXCommand:USER [mathon_k]   GROUP [ - ] TOKEN [] APP [oo_20140328]   JOB [0000028-140328083804680-Oozie的-oozi-W]   ACTION [0000028-140328083804680-Oozie的-oozi-W @杀]   [ 0000028-140328083804680-oozie-oozi-W @ kill ]在DB中更新了操作!   2014-03-28 15:11:45,626警告   org.apache.oozie.command.coord.CoordActionUpdateXCommand:   USER [mathon_k] GROUP [ - ] TOKEN [] APP [oo_20140328]   工作[0000028-140328083804680-oozie-oozi-W]行动[ - ] E1100:命令   执行前,条件不成立,[,coord action为null],   错误代码:E1100

但是当我将shell脚本更改为以下内容时,它运行良好。

.#!/bin/bash

.#hadoop fs -mkdir /user/mathon_k/shell_dir

echo "hello "

2 个答案:

答案 0 :(得分:1)

如果你想使用mkdir - 请使用workflow HDFS action

如果你想运行bash - 似乎工作正常

答案 1 :(得分:0)

由于安全问题,我们还遇到了在Oozie中发出hadoop shell命令的问题。 您使用的是哪个版本的Hadoop?哪个版本oozie?

对于初学者,尝试将HADOOP_USER_NAME设置为您希望hadoop命令在执行之前执行的用户的名称。