Question

我有一个春季批处理工作，我想做以下...

Step 1 - 
   Tasklet - Create a list of dates, store the list of dates in the job execution context.

Step 2 - 
   JDBC Item Reader - Get list of dates from job execution context.
                      Get element(0) in dates list. Use is as input for jdbc query. 
                      Store element(0) date is job execution context 
                      Remove element(0) date from list of dates
                      Store element(0) date in job execution context                 
   Flat File Item Writer - Get element(0) date from job execution context and use for file name.

Then using a job listener repeat step 2 until no remaining dates in the list of dates.

我已经创建了这个工作，它可以用于第一次执行第2步。但是第2步并没有像我想要的那样重复。我知道这一点，因为当我通过我的代码进行调试时，它只会在第2步的初始运行时中断。

然而它确实继续向我发送如下消息，就好像它正在运行第2步，即使我知道它不是。

2016-08-10 22:20:57.842  INFO 11784 --- [           main] o.s.batch.core.job.SimpleStepHandler     : Duplicate step [readStgDbAndExportMasterListStep] detected in execution of job=[exportMasterListCsv]. If either step fails, both will be executed again on restart.
2016-08-10 22:20:57.846  INFO 11784 --- [           main] o.s.batch.core.job.SimpleStepHandler     : Executing step: [readStgDbAndExportMasterListStep]

这最终会以永无止境的循环结束。

有人可以帮我解决或提出为什么我的stpe 2只运行一次？

提前致谢

我为我的代码添加了两个指向PasteBin的链接，以免污染此帖子。

http://pastebin.com/QhExNikm（工作配置）

http://pastebin.com/sscKKWRk（普通工作配置）

http://pastebin.com/Nn74zTpS（步骤执行监听器）

Answer 1

根据您的问题和您的代码，我根据您检索的日期数量（这在实际作业开始之前发生）推断出，您将执行一个步骤，查看您拥有日期的次数。

我建议改变设计。创建一个java类，它将日期作为列表，并根据该列表，您将动态创建您的步骤。像这样：

@EnableBatchProcessing
public class JobConfig {

    @Autowired
    private JobBuilderFactory jobBuilderFactory;

    @Autowired
    private StepBuilderFactory stepBuilderFactory;  

    @Autowired
    private JobDatesCreator jobDatesCreator;

    @Bean
    public Job executeMyJob() {
        List<Step> steps = new ArrayList<Step>();
        for (String date : jobDatesCreator.getDates()) {
            steps.add(createStep(date));
        }

        return jobBuilderFactory.get("executeMyJob")
                .start(createParallelFlow(steps))
                .end()
                .build();       
    }

    private Step createStep(String date){
        return stepBuilderFactory.get("readStgDbAndExportMasterListStep" + date)
                .chunk(your_chunksize)
                .reader(your_reader)
                .processor(your_processor)
                .writer(your_writer)                                
                .build();       
    }   

    private Flow createParallelFlow(List<Step> steps) {
        SimpleAsyncTaskExecutor taskExecutor = new SimpleAsyncTaskExecutor();
        // max multithreading = -1, no multithreading = 1, smart size = steps.size()
        taskExecutor.setConcurrencyLimit(1); 

        List<Flow> flows = steps.stream()
                .map(step -> new FlowBuilder<Flow>("flow_" + step.getName()).start(step).build())
                .collect(Collectors.toList());

        return new FlowBuilder<SimpleFlow>("parallelStepsFlow")
                .split(taskExecutor)
                .add(flows.toArray(new Flow[flows.size()]))
                .build();
    }  
}

编辑：添加了“jobParameter”输入（也略有不同的方法）

在类路径的某处添加以下示例.properties文件：

sql.statement="select * from awesome"

并将以下注释添加到JobDatesCreator类

@PropertySource("classpath:example.properties")

您也可以提供特定的sql语句作为命令行参数。从春季文档：

您可以使用特定的命令行开关启动（例如java -jar app.jar --name =“Spring”）。

有关详细信息，请参阅http://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html

获取日期的类（为什么要使用tasklet？）：

@PropertySource("classpath:example.properties")
public class JobDatesCreator {

    @Value("${sql.statement}")
    private String sqlStatement;

    @Autowired
    private CommonExportFromStagingDbJobConfig commonJobConfig; 

    private List<String> dates; 

    @PostConstruct
    private void init(){
        // Execute your logic here for getting the data you need.
        JdbcTemplate jdbcTemplate = new JdbcTemplate(commonJobConfig.onlineStagingDb);
        // acces to your sql statement provided in a property file or as a command line argument
        System.out.println("This is the sql statement I provided in my external property: " + sqlStatement);

        // for now..
        dates = new ArrayList<>();
        dates.add("date 1");
        dates.add("date 2");
    }

    public List<String> getDates() {
        return dates;
    }

    public void setDates(List<String> dates) {
        this.dates = dates;
    }
}

我还注意到你有很多重复的代码，你很容易重构。现在，对于每个作家，你都有这样的东西：

@Bean
public FlatFileItemWriter<MasterList> division10MasterListFileWriter() {
    FlatFileItemWriter<MasterList> writer = new FlatFileItemWriter<>();
    writer.setResource(new FileSystemResource(new File(outDir, MerchHierarchyConstants.DIVISION_NO_10 )));
    writer.setHeaderCallback(masterListFlatFileHeaderCallback());
    writer.setLineAggregator(masterListFormatterLineAggregator());
    return writer;
}

考虑使用类似的东西：

public FlatFileItemWriter<MasterList> divisionMasterListFileWriter(String divisionNumber) {
    FlatFileItemWriter<MasterList> writer = new FlatFileItemWriter<>();
    writer.setResource(new FileSystemResource(new File(outDir, divisionNumber )));
    writer.setHeaderCallback(masterListFlatFileHeaderCallback());
    writer.setLineAggregator(masterListFormatterLineAggregator());
    return writer;
}

由于并非所有代码都可以正确复制您的问题，因此这个答案是解决您问题的建议/指示。

Spring批处理重复步骤以永无止境的循环结束

1 个答案: