Question

我正在使用Spring Batch 4.1.2.RELEASE。我对平行步骤有疑问（使用拆分流）。当SplitFlow的concurrencyLimit（或ThreadPoolTaskExecutor.corePoolSize）小于A拆分流的步骤数时。作业永不停止，也不会引发异常。

我知道解决方案是增加concurrencyLimit或减少每个流程中的步骤数。但是我想确定作业的TaskExecutor和任务的TaskExecutor是否存在问题，或者我的代码是否错误。

在不考虑SplitFlow的情况下，我发现如果提交给jobLauncher的Jobs（尽可能简单）的数量大于其TaskExecutor.corePoolSize（假设1），则该作业将被一一执行。这是预期的结果。

   @Bean
    public TaskExecutor taskExecutor() {
        SimpleAsyncTaskExecutor executor = new SimpleAsyncTaskExecutor("tsk-Exec-");
        executor.setConcurrencyLimit(2);
        return executor;
    }

@SpringBootApplication
@EnableBatchProcessing
@EnableWebMvc
public class BatchJobApplication {
    public static void main(String[] args) {
        SpringApplication.run(BatchJobApplication.class, args);
    }
}

下面的代码创建一个作业，包含一个带有4个tasklet步骤的拆分流。

@Autowired
private TaskExecutor taskExecutor;


public JobExecution experiment(Integer flowId) {
        String dateFormat = LocalDate.now(ZoneId.of("+8")).format(DateTimeFormatter.BASIC_ISO_DATE);
        JobBuilder job1 = this.jobBuilderFactory.get("Job_" + flowId + "_" + dateFormat);

        List<TaskletStep> taskletSteps = Lists.newArrayList();
        for (int i = 0; i < 4; i++) {
            taskletSteps.add(this.stepBuilderFactory.get("step:" + i).tasklet(
                    (contribution, chunkContext) -> {
                        Thread.sleep(3000);
                        return RepeatStatus.FINISHED;
                    }).build());
        }

        JobExecution run = null;
        FlowBuilder.SplitBuilder<SimpleFlow> splitFlow = new FlowBuilder<SimpleFlow>("splitFlow").split(taskExecutor);
        FlowBuilder<SimpleFlow> lastFlowNode = null;
        for (TaskletStep taskletStep : taskletSteps) {
            SimpleFlow singleNode = new FlowBuilder<SimpleFlow>("async-fw-" + taskletStep.getName()).start(taskletStep).build();
            lastFlowNode = splitFlow.add(singleNode);
        }

        Job build = job1.start(lastFlowNode.end()).build().build();
        JobParametersBuilder jobParametersBuilder = new JobParametersBuilder();
        jobParametersBuilder.addDate("parameterGenerated", new Date());
        try {
            run = jobLauncher.run(build, jobParametersBuilder.toJobParameters());
        } catch (JobExecutionAlreadyRunningException e) {
            e.printStackTrace();
        } catch (JobRestartException e) {
            e.printStackTrace();
        } catch (JobInstanceAlreadyCompleteException e) {
            e.printStackTrace();
        } catch (JobParametersInvalidException e) {
            e.printStackTrace();
        }
        return run;
    }

现在已被阻止。

2019-07-29 18:08:10.321  INFO 24416 --- [     job-Exec-1] o.s.b.c.l.support.SimpleJobLauncher      : Job: [FlowJob: [name=Job_2124_20190729]] launched with the following parameters: [{parameterGenerated=1564394890193}]
2019-07-29 18:08:13.392 DEBUG 24416 --- [     job-Exec-1] cTaskExecutor$ConcurrencyThrottleAdapter : Entering throttle at concurrency count 0
2019-07-29 18:08:13.393 DEBUG 24416 --- [     job-Exec-1] cTaskExecutor$ConcurrencyThrottleAdapter : Entering throttle at concurrency count 1
2019-07-29 18:08:13.393 DEBUG 24416 --- [     tsk-Exec-2] cTaskExecutor$ConcurrencyThrottleAdapter : Concurrency count 2 has reached limit 2 - blocking
2019-07-29 18:08:13.425  INFO 24416 --- [     tsk-Exec-1] o.s.batch.core.job.SimpleStepHandler     : Executing step: [step:3]
2019-07-29 18:08:16.466 DEBUG 24416 --- [     tsk-Exec-1] cTaskExecutor$ConcurrencyThrottleAdapter : Returning from throttle at concurrency count 1
2019-07-29 18:08:16.466 DEBUG 24416 --- [     tsk-Exec-2] cTaskExecutor$ConcurrencyThrottleAdapter : Entering throttle at concurrency count 1
2019-07-29 18:08:16.466 DEBUG 24416 --- [     tsk-Exec-2] cTaskExecutor$ConcurrencyThrottleAdapter : Concurrency count 2 has reached limit 2 - blocking
2019-07-29 18:08:16.484  INFO 24416 --- [     tsk-Exec-3] o.s.batch.core.job.SimpleStepHandler     : Executing step: [step:2]
2019-07-29 18:08:19.505 DEBUG 24416 --- [     tsk-Exec-3] cTaskExecutor$ConcurrencyThrottleAdapter : Returning from throttle at concurrency count 1
2019-07-29 18:08:19.505 DEBUG 24416 --- [     tsk-Exec-2] cTaskExecutor$ConcurrencyThrottleAdapter : Entering throttle at concurrency count 1
2019-07-29 18:08:19.506 DEBUG 24416 --- [     tsk-Exec-4] cTaskExecutor$ConcurrencyThrottleAdapter : Concurrency count 2 has reached limit 2 - blocking

Answer 1

我想昨天晚上我找到了答案。

当SplitFlow中的TaskExecutor具有一些concurrencyLimit或ThreadPoolTaskExecutor.corePoolSize时。根据代码，很可能发生所有线程都被future.get（）阻塞的情况，但是没有可用的线程有机会运行taskletStep。

//SplitState.java:114

results.add(task.get());

此外，由TaskExecutor在JobLauncher中创建的线程不必等待将来的结果。因此TaskExcutor总是有足够的可用线程来接受作业，而无需等待任何条件。

当TaskExecutor concurrencyLimit小于流程步骤数时，作业将被阻止

1 个答案: