多线程步骤正在运行太多线程

时间:2019-07-02 15:37:42

标签: spring-boot spring-batch

我创建了一个多线程步骤,最大线程限制为10;

private static final int THREAD_NUMBER = 10;

@Bean
public TaskExecutor taskExecutor(){
    SimpleAsyncTaskExecutor simpleAsyncTaskExecutor = new SimpleAsyncTaskExecutor("batch-thread");
    simpleAsyncTaskExecutor.setConcurrencyLimit(THREAD_NUMBER);
    return simpleAsyncTaskExecutor;
}


@Bean
protected Tasklet readHeader() {
    EkuampVorsatzReader ekuampVorsatzReader = new EkuampVorsatzReader();
    return ekuampVorsatzReader;
}

@Bean
public Step stepGetPriceFileHeader(){
    return stepBuilderFactory.get("batch-job_price_file-step-get-pricefile-header").tasklet(readHeader()).build();
}

@Bean
public Step stepGetPriceFileData() {
    return stepBuilderFactory.get("batch-job_price_file-step-get-pricefile-data")
            .<ImportLogEntity, ImportLogEntity>chunk(50)
            .reader(priceFileReader).listener(priceFileReaderListener)
            .writer(priceFileWriter)
            .listener(priceFileStepListener)
            .taskExecutor(taskExecutor())
            .throttleLimit(THREAD_NUMBER)
            .build();
}

@Bean
public Job batchJobPriceFile() {
    Job job = jobBuilderFactory.get("batch-job_price_file")
            .start(stepGetPriceFileHeader())
            .next(stepGetPriceFileData())
            .listener(jobExecutionListener)
            .build();
    return job;
}

在处理了200万条记录文件之后,即使将限制设置为10个线程,我也可以在日志文件中看到创建了太多线程。 例如:INFO [batch-thread35348] 你能告诉我为什么吗?谢谢。

  

2019-07-02T17:02:29,896 8129857信息[batch-thread35348] com.db.wmdl.pricefile.job.batch.listener.PriceFileReaderListener([])-PriceFileReaderListener-beforeRead   2019-07-02T17:02:30,147 8130108信息[batch-thread35341] com.db.wmdl.pricefile.job.batch.listener.PriceFileWriterListener([])-PriceFileWriterListener-afterWrite;清单大小:50   2019-07-02T17:02:30,191 8130152信息[batch-thread35343] com.db.wmdl.pricefile.job.batch.listener.PriceFileWriterListener([])-PriceFileWriterListener-afterWrite;清单大小:50   2019-07-02T17:02:30,219 8130180信息[batch-thread35342] com.db.wmdl.pricefile.job.batch.listener.PriceFileWriterListener([])-PriceFileWriterListener-afterWrite;清单大小:50   2019-07-02T17:02:30,340 8130301信息[batch-thread35344] com.db.wmdl.pricefile.job.batch.listener.PriceFileWriterListener([])-PriceFileWriterListener-afterWrite;清单大小:50   2019-07-02T17:02:30,504 8130465信息[batch-thread35345] com.db.wmdl.pricefile.job.batch.listener.PriceFileWriterListener([])-PriceFileWriterListener-afterWrite;清单大小:50   2019-07-02T17:02:30,513 8130474信息[batch-thread35349] com.db.wmdl.pricefile.job.batch.listener.PriceFileReaderListener([])-PriceFileReaderListener-beforeRead   2019-07-02T17:02:30,513 8130474信息[batch-thread35350] com.db.wmdl.pricefile.job.batch.listener.PriceFileReaderListener([])-PriceFileReaderListener-beforeRead   2019-07-02T17:02:30,513 8130474信息[batch-thread35351] com.db.wmdl.pricefile.job.batch.listener.PriceFileReaderListener([])-PriceFileReaderListener-beforeRead   2019-07-02T17:02:30,513 8130474信息[batch-thread35352] com.db.wmdl.pricefile.job.batch.listener.PriceFileReaderListener([])-PriceFileReaderListener-beforeRead   2019-07-02T17:02:30,513 8130474信息[batch-thread35354] com.db.wmdl.pricefile.job.batch.listener.PriceFileReaderListener([])-PriceFileReaderListener-beforeRead   2019-07-02T17:02:30,513 8130474信息[batch-thread35353] com.db.wmdl.pricefile.job.batch.listener.PriceFileReaderListener([])-PriceFileReaderListener-beforeRead   2019-07-02T17:02:30,513 8130474信息[batch-thread35355] com.db.wmdl.pricefile.job.batch.listener.PriceFileReaderListener([])-PriceFileReaderListener-beforeRead   2019-07-02T17:02:30,586 8130547信息[QuartzScheduler_Worker-1] com.db.wmdl.pricefile.job.batch.listener.PriceFileJobListener([])-SpringBatchJob [入站编号:56435] [spring-batch-job -execution-id:21]导入完成

1 个答案:

答案 0 :(得分:0)

SimpleAsyncTaskExecutor不重用线程。以下是其Javadoc的摘录:

  

注意:此实现不重用线程!请考虑使用线程池TaskExecutor实现,特别是对于执行大量短期任务。

根据您的情况,您可以使用ThreadPoolTaskExecutor并将其maxPoolSize设置为10。