带踩响动作的Oozie工作挂起

时间:2018-10-28 10:31:36

标签: cloudera oozie

在cloudera.quickstart虚拟机中,我正在学习oozie,并尝试通过sqoop操作运行简单的工作流程。 当我执行工作时,它似乎挂起了,我不知道那里发生了什么。 我没有注意到错误,还是缺少通知网址是问题所在?

2018-10-28 03:23:26,481  INFO JPAService:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[sqoop-wf] JOB[0000003-181027074351697-oozie-oozi-W] ACTION[] No results found
2018-10-28 03:23:26,575  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[sqoop-wf] JOB[0000003-181027074351697-oozie-oozi-W] ACTION[0000003-181027074351697-oozie-oozi-W@:start:] Start action [0000003-181027074351697-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2018-10-28 03:23:26,576  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[sqoop-wf] JOB[0000003-181027074351697-oozie-oozi-W] ACTION[0000003-181027074351697-oozie-oozi-W@:start:] [***0000003-181027074351697-oozie-oozi-W@:start:***]Action status=DONE
2018-10-28 03:23:26,576  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[sqoop-wf] JOB[0000003-181027074351697-oozie-oozi-W] ACTION[0000003-181027074351697-oozie-oozi-W@:start:] [***0000003-181027074351697-oozie-oozi-W@:start:***]Action updated in DB!
2018-10-28 03:23:26,690  INFO JPAService:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[sqoop-wf] JOB[0000003-181027074351697-oozie-oozi-W] ACTION[0000003-181027074351697-oozie-oozi-W@:start:] No results found
2018-10-28 03:23:26,740  INFO WorkflowNotificationXCommand:520 - SERVER[quickstart.cloudera] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000003-181027074351697-oozie-oozi-W] ACTION[0000003-181027074351697-oozie-oozi-W@:start:] No Notification URL is defined. Therefore nothing to notify for job 0000003-181027074351697-oozie-oozi-W@:start:
2018-10-28 03:23:26,740  INFO WorkflowNotificationXCommand:520 - SERVER[quickstart.cloudera] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000003-181027074351697-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000003-181027074351697-oozie-oozi-W
2018-10-28 03:23:26,812  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[sqoop-wf] JOB[0000003-181027074351697-oozie-oozi-W] ACTION[0000003-181027074351697-oozie-oozi-W@sqoop-node] Start action [0000003-181027074351697-oozie-oozi-W@sqoop-node] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]

工作流程定义为

<?xml version="1.0" encoding="UTF-8"?>

<workflow-app xmlns="uri:oozie:workflow:0.2" name="sqoop-wf">
    <start to="sqoop-node"/>

    <action name="sqoop-node">
        <sqoop xmlns="uri:oozie:sqoop-action:0.2">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <prepare>
                <delete path="${nameNode}/user/${wf:user()}/${examplesRoot}/output-data/sqoop"/>
                <mkdir path="${nameNode}/user/${wf:user()}/${examplesRoot}/output-data"/>
            </prepare>
            <configuration>
                <property>
                    <name>mapred.job.queue.name</name>
                    <value>${queueName}</value>
                </property>
            </configuration>
            <command>import --connect jdbc:mysql://localhost:3306/retail_db --username=retail_dba --password cloudera \
--table departments --as-textfile --target-dir /user/cloudera/departments</command>
        </sqoop>
        <ok to="end"/>
        <error to="fail"/>
    </action>

    <kill name="fail">
        <message>Sqoop failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>

属性文件

nameNode=hdfs://localhost:8020
jobTracker=localhost:8023
queueName=default
examplesRoot=questions_oozie/q59

oozie.use.system.libpath=true

oozie.wf.application.path=${nameNode}/user/${user.name}/${examplesRoot}/

添加:我创建了另一个oozie工作流,这次使用HUE和Pig脚本。因此,工作流是由HUE生成的,不应包含错误。我认为问题与我的集群配置有关?

2018-10-28 07:01:53,110  INFO JPAService:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My Workflow] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[] No results found
2018-10-28 07:01:53,240  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My Workflow] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[0000007-181027074351697-oozie-oozi-W@:start:] Start action [0000007-181027074351697-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2018-10-28 07:01:53,244  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My Workflow] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[0000007-181027074351697-oozie-oozi-W@:start:] [***0000007-181027074351697-oozie-oozi-W@:start:***]Action status=DONE
2018-10-28 07:01:53,244  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My Workflow] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[0000007-181027074351697-oozie-oozi-W@:start:] [***0000007-181027074351697-oozie-oozi-W@:start:***]Action updated in DB!
2018-10-28 07:01:53,428  INFO JPAService:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My Workflow] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[0000007-181027074351697-oozie-oozi-W@:start:] No results found
2018-10-28 07:01:53,478  INFO WorkflowNotificationXCommand:520 - SERVER[quickstart.cloudera] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[0000007-181027074351697-oozie-oozi-W@:start:] No Notification URL is defined. Therefore nothing to notify for job 0000007-181027074351697-oozie-oozi-W@:start:
2018-10-28 07:01:53,479  INFO WorkflowNotificationXCommand:520 - SERVER[quickstart.cloudera] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000007-181027074351697-oozie-oozi-W
2018-10-28 07:01:53,564  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My Workflow] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[0000007-181027074351697-oozie-oozi-W@pig-ca9a] Start action [0000007-181027074351697-oozie-oozi-W@pig-ca9a] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2018-10-28 07:01:55,590  INFO PigActionExecutor:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My Workflow] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[0000007-181027074351697-oozie-oozi-W@pig-ca9a] checking action, hadoop job ID [job_1540651330075_0018] status [RUNNING]
2018-10-28 07:01:55,600  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My Workflow] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[0000007-181027074351697-oozie-oozi-W@pig-ca9a] [***0000007-181027074351697-oozie-oozi-W@pig-ca9a***]Action status=RUNNING
2018-10-28 07:01:55,600  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My Workflow] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[0000007-181027074351697-oozie-oozi-W@pig-ca9a] [***0000007-181027074351697-oozie-oozi-W@pig-ca9a***]Action updated in DB!
2018-10-28 07:01:55,666  INFO WorkflowNotificationXCommand:520 - SERVER[quickstart.cloudera] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[0000007-181027074351697-oozie-oozi-W@pig-ca9a] No Notification URL is defined. Therefore nothing to notify for job 0000007-181027074351697-oozie-oozi-W@pig-ca9a
2018-10-28 07:05:30,298  INFO CallbackServlet:520 - SERVER[quickstart.cloudera] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[0000007-181027074351697-oozie-oozi-W@pig-ca9a] callback for action [0000007-181027074351697-oozie-oozi-W@pig-ca9a]
2018-10-28 07:05:30,696  INFO PigActionExecutor:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My Workflow] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[0000007-181027074351697-oozie-oozi-W@pig-ca9a] Hadoop Jobs launched : [job_1540651330075_0020,job_1540651330075_0021]
2018-10-28 07:05:30,701  INFO PigActionExecutor:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My Workflow] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[0000007-181027074351697-oozie-oozi-W@pig-ca9a] action completed, external ID [job_1540651330075_0018]
2018-10-28 07:05:30,725  WARN PigActionExecutor:523 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My Workflow] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[0000007-181027074351697-oozie-oozi-W@pig-ca9a] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.PigMain], exit code [2]
2018-10-28 07:05:30,772  INFO ActionEndXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My Workflow] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[0000007-181027074351697-oozie-oozi-W@pig-ca9a] ERROR is considered as FAILED for SLA
2018-10-28 07:05:30,834  INFO JPAService:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My Workflow] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[0000007-181027074351697-oozie-oozi-W@pig-ca9a] No results found
2018-10-28 07:05:30,875  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My Workflow] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[0000007-181027074351697-oozie-oozi-W@Kill] Start action [0000007-181027074351697-oozie-oozi-W@Kill] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2018-10-28 07:05:30,875  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My Workflow] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[0000007-181027074351697-oozie-oozi-W@Kill] [***0000007-181027074351697-oozie-oozi-W@Kill***]Action status=DONE
2018-10-28 07:05:30,875  INFO ActionStartXCommand:520 - SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[My Workflow] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[0000007-181027074351697-oozie-oozi-W@Kill] [***0000007-181027074351697-oozie-oozi-W@Kill***]Action updated in DB!
2018-10-28 07:05:31,069  INFO WorkflowNotificationXCommand:520 - SERVER[quickstart.cloudera] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[0000007-181027074351697-oozie-oozi-W@Kill] No Notification URL is defined. Therefore nothing to notify for job 0000007-181027074351697-oozie-oozi-W@Kill
2018-10-28 07:05:31,071  INFO WorkflowNotificationXCommand:520 - SERVER[quickstart.cloudera] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000007-181027074351697-oozie-oozi-W
2018-10-28 07:05:31,072  INFO WorkflowNotificationXCommand:520 - SERVER[quickstart.cloudera] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000007-181027074351697-oozie-oozi-W] ACTION[0000007-181027074351697-oozie-oozi-W@pig-ca9a] No Notification URL is defined. Therefore nothing to notify for job 0000007-181027074351697-oozie-oozi-W@pig-ca9a

0 个答案:

没有答案