Question

我正在尝试设置oozie和sqoop工作流程（我想将mySql数据备份到我的hdfs中）。但是当我尝试开始工作时，我感到困惑。

我正在使用hadoop2（工作hdfs节点），oozie的最后一个版本。我在我的计算机上安装了oozie服务器（我想在部署它之前测试它）和hdfs配置（oozie conf /上的core-site.xml，hdfs-site.xml，yarn-site.xml，mapred-site.xml） haddop-conf dir），以及我在服务器上的hdfs。

我已经做了一个基本的工作流程（测试目的，我只是想看看sqoop是否正常工作），如下所示：

<workflow-app name="Sqoop" xmlns="uri:oozie:workflow:0.4">
    <start to="Sqoop"/>
    <action name="Sqoop">
        <sqoop xmlns="uri:oozie:sqoop-action:0.2">
         <job-tracker>yarn.resourcemanager.address:8040</job-tracker>
          <name-node>hdfs://hdfs-server:54310</name-node>
          <command>job --list</command>
        </sqoop>
        <ok to="end"/>
        <error to="kill"/>
    </action>
    <kill name="kill">
        <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>

我把这个工作流程放到我的hdfs中。我已经创建了一个java代码来开始我的工作：

OozieClient wc = new OozieClient("http://localhost:11000/oozie");

        Properties conf = wc.createConfiguration();
        conf.setProperty( OozieClient.APP_PATH, "hdfs://hdfs_server:54310/hive/testSqoop/sqoop-workflow.xml" );
        conf.setProperty("queueName", "default");

        try {
            String jobId = wc.run(conf);
            System.out.println("Workflow job submitted");

            while (wc.getJobInfo(jobId).getStatus() == WorkflowJob.Status.RUNNING) {
                System.out.println("Workflow job running ...");
                System.out.println("..." + wc.getJobInfo(jobId).getStatus().toString() );
                Thread.sleep(10 * 1000);
            }
            System.out.println("Workflow job completed ...");
            System.out.println(wc.getJobInfo(jobId));
        } catch (Exception r) {
            r.printStackTrace();
        }

在Oozie网络界面中，我可以看到我的工作正在运行

2013-05-28 12:42:30,004  INFO ActionStartXCommand:539 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[0000000-130528124140043-oozie-anth-W@:start:] Start action [0000000-130528124140043-oozie-anth-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2013-05-28 12:42:30,008  WARN ActionStartXCommand:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[0000000-130528124140043-oozie-anth-W@:start:] [***0000000-130528124140043-oozie-anth-W@:start:***]Action status=DONE
2013-05-28 12:42:30,009  WARN ActionStartXCommand:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[0000000-130528124140043-oozie-anth-W@:start:] [***0000000-130528124140043-oozie-anth-W@:start:***]Action updated in DB!
2013-05-28 12:42:30,192  INFO ActionStartXCommand:539 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[0000000-130528124140043-oozie-anth-W@Sqoop] Start action [0000000-130528124140043-oozie-anth-W@Sqoop] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2013-05-28 12:42:31,389  WARN SqoopActionExecutor:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[0000000-130528124140043-oozie-anth-W@Sqoop] credentials is null for the action
2013-05-28 12:42:42,942  INFO SqoopActionExecutor:539 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[0000000-130528124140043-oozie-anth-W@Sqoop] checking action, external ID [job_1369126414383_0003] status [RUNNING]
2013-05-28 12:42:42,945  WARN ActionStartXCommand:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[0000000-130528124140043-oozie-anth-W@Sqoop] [***0000000-130528124140043-oozie-anth-W@Sqoop***]Action status=RUNNING
2013-05-28 12:42:42,946  WARN ActionStartXCommand:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[0000000-130528124140043-oozie-anth-W@Sqoop] [***0000000-130528124140043-oozie-anth-W@Sqoop***]Action updated in DB!
2013-05-28 12:47:43,034  INFO KillXCommand:539 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[-] STARTED WorkflowKillXCommand for jobId=0000000-130528124140043-oozie-anth-W
2013-05-28 12:47:43,328  WARN CoordActionUpdateXCommand:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[-] E1100: Command precondition does not hold before execution, [, coord action is null], Error Code: E1100
2013-05-28 12:47:43,328  INFO KillXCommand:539 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[-] ENDED WorkflowKillXCommand for jobId=0000000-130528124140043-oozie-anth-W

当我检查纱线网接口时，我可以看到我的工作，但状态为FAILED

Application application_1369126414383_0003 failed 1 times due to AM Container for appattempt_1369126414383_0003_000001 exited with exitCode: 1 due to: .Failing this attempt.. Failing the application.

我真的不知道出了什么问题。我需要你的建议。

谢谢〜

Answer 1

您必须检查作业日志：

$ oozie job -log <coord_job_id>

了解正在发生的事情。

Oozie和sqoop配置问题

1 个答案: