Question

如何在java api中将作业流程设置为“保持活动状态”，就像我执行命令一样：

elastic-mapreduce --create --alive ...

我尝试添加withKeepJobFlowAlivewhenNoSteps（true），但这仍然会在步骤失败时关闭作业流（例如，如果我提交了一个坏jar）

Answer 1

您需要设置withActionOnFailure以让API知道步骤失败时要执行的操作，并且必须逐步设置。

您的withActionOnFailure("TERMINATE_JOB_FLOW")必须StepConfig。将它们更改为withActionOnFailure("CANCEL_AND_WAIT")。

以下是使用从here获取的Java API启动集群的完整代码，仅替换所需内容：

AWSCredentials credentials = new BasicAWSCredentials(accessKey, secretKey);
   AmazonElasticMapReduceClient emr = new AmazonElasticMapReduceClient(credentials);

   StepFactory stepFactory = new StepFactory();

   StepConfig enabledebugging = new StepConfig()
       .withName("Enable debugging")
       .withActionOnFailure("CANCEL_AND_WAIT") //here is the change
       .withHadoopJarStep(stepFactory.newEnabledebuggingStep());

   StepConfig installHive = new StepConfig()
       .withName("Install Hive")
       .withActionOnFailure("CANCEL_AND_WAIT") //here is the change
       .withHadoopJarStep(stepFactory.newInstallHiveStep());

   RunJobFlowRequest request = new RunJobFlowRequest()
       .withName("Hive Interactive")
       .withSteps(enabledebugging, installHive)
       .withLogUri("s3://myawsbucket/")
       .withInstances(new JobFlowInstancesConfig()
           .withEc2KeyName("keypair")
           .withHadoopVersion("0.20")
           .withInstanceCount(5)
           .withKeepJobFlowAliveWhenNoSteps(true)
           .withMasterInstanceType("m1.small")
           .withSlaveInstanceType("m1.small"));

   RunJobFlowResult result = emr.runJobFlow(request);

弹性映射减少java api中的“keep alive”规范

1 个答案: