发生异常时ClassifierCompositeItemWriter回滚,但数据已部分提交给数据库

时间:2019-06-19 22:18:01

标签: spring spring-boot transactions spring-batch spring-jdbc

我正在使用ClassifierCompositeItemWriter将不同类型的寄存器插入一个固定长度的平面文件中,并将其写入到  具有多个JdbcBatchItemWriters的postgres数据库,每个数据库位于一个不同的表中,所有这些都在一个步骤中,然后坚持执行春季批处理作业,它可以正常工作,但是在激活事务时,它们将在发生异常的情况下回滚。

例如,我有一个32行的平面文件,其中1行是标题记录,然后将其插入标题表中,然后有30条常规记录和1条页脚记录(按顺序),然​​后在29条记录中常规的它失败了,并带有数据库转换异常(为测试而创建的错误),然后以作业的失败状态结束,没关系,但是当我查看数据库时,我发现头的记录为1,常规数据的记录为29 (除了一个有错误的错误),并且没有页脚记录,但是我希望该事务回滚该记录中有1条记录为标头,其余29条记录,但是在异常回滚之后它们仍在数据库中。

我不知道我是否做错了,春季批处理中的事务无法正常工作,或者我的配置有误还是什么。

这是ClassifierCompositeItemWriter的代码,一个项目编写器的代码与此类似:

public ClassifierCompositeItemWriter<DTOBase> altasOffWriterClassifier(DataSource dataSource) {

    BackToBackPatternClassifier classifier = new BackToBackPatternClassifier();
    classifier.setRouterDelegate(dtoWriterClassifier);
    classifier.setMatcherMap(new HashMap<String, JdbcBatchItemWriter<? extends DTOBase>>() {
        private static final long serialVersionUID = -1247536568421993759L;
    {
        put(DTOHeader.class.getTypeName(), headerWriter());
        put(DTOData.class.getTypeName(), dataWriter());
        put(DTOFooter.class.getTypeName(), footerWriterFin());
    }});

    ClassifierCompositeItemWriter<DTOBase> writer = new ClassifierCompositeItemWriter<>();
    writer.setClassifier(classifier);

    return writer;
}

@Bean
public JdbcBatchItemWriter<DTOAltaOFF> altaOffWriter() {
    return new JdbcBatchItemWriterBuilder<DTOData>()
         .dataSource(dataSource)
         .sql("insert into tabla(ticket, identificador, fecha_alta_operacion, "
                + " ordenante, numero, moneda, cif, importe_emisor, "
                + " estado, telefono_destino, fecha_caducidad_hal, concepto, cabecera_num_orden_fichero) "
                + " VALUES (:ticket,:identificador,to_timestamp(:fechaAltaOperacion,'DDMMYYYYHH24MISS'), "
                + " :ordenante,:numero,:moneda,:cif,(cast(:importeEmisor as double precision)/100), "
                + " :estado,:telefonoDestino,to_timestamp(:fechaCaducidadHal,'DDMMYYYYHH24MISS'),:concepto,:idCabecera) ")
         .itemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<>()) 
         .build();     
}

我的配置类:

@Configuration
@EnableBatchProcessing
@Import({ DataSourceConfig.class })
@PropertySource("classpath:batch.properties")
@ComponentScan({ "..."})
public class BatchConfiguration {

@Autowired
public JobBuilderFactory jobBuilderFactory;

@Autowired
public StepBuilderFactory stepBuilderFactory;

@Autowired
public JobRepository jobRepository;

@Autowired
public DataSource dataSource;

@Bean
public JdbcTemplate getJdbcTemplate() {
    return new JdbcTemplate(dataSource);
}

@Bean
public TaskExecutor taskExecutor() {
    return new SimpleAsyncTaskExecutor();
}

数据源

@Configuration
@EnableTransactionManagement
public class DataSourceConfig {

...some @Value...   
    @Bean(name = "dataSource")
    public DriverManagerDataSource dataSource() {
        DriverManagerDataSource dataSource = new DriverManagerDataSource();
        dataSource.setDriverClassName(driverClassName);
        dataSource.setUrl(datasourceUrl);
        dataSource.setUsername(usuario);
        dataSource.setPassword(clave);
        return dataSource;
    }
}

配置:

@Configuration
@EnableBatchProcessing
@Import({ DataSourceConfig.class })
@PropertySource("classpath:batch.properties")
@ComponentScan({ "..."})
public class BatchConfiguration {

    @Autowired
    public JobBuilderFactory jobBuilderFactory;

    @Autowired
    public StepBuilderFactory stepBuilderFactory;

    @Autowired
    public JobRepository jobRepository;

    @Autowired
    public DataSource dataSource;

    @Bean
    public JdbcTemplate getJdbcTemplate() {
        return new JdbcTemplate(dataSource);
    }

    @Bean
    public TaskExecutor taskExecutor() {
        return new SimpleAsyncTaskExecutor();
    }
}

自定义:

@Component
@EnableTransactionManagement
public class CustomBatchConfigurer extends DefaultBatchConfigurer {

   private final TaskExecutor taskExecutor;

    public CustomBatchConfigurer(TaskExecutor taskExecutor) {
        this.taskExecutor = taskExecutor;
    }

    @Override
    protected JobLauncher createJobLauncher() throws Exception {
        SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
        jobLauncher.setJobRepository(getJobRepository());
        jobLauncher.setTaskExecutor(this.taskExecutor);
        jobLauncher.afterPropertiesSet();
        return jobLauncher;
    }

    @Autowired
    private DataSource dataSource;

    @Override
    public PlatformTransactionManager getTransactionManager() {
        DataSourceTransactionManager tm = new DataSourceTransactionManager();
        tm.setDataSource(dataSource);
        return tm;
    }
}

任何帮助都会很棒。

1 个答案:

答案 0 :(得分:0)

就像@MahmoudBenHassine在评论中所写的那样,它是块大小,我以这种方式添加了它

@Bean
public Step Step1(@Qualifier("xyzWriterClassifier") ItemWriter<DTOxyz> writer) throws Exception {
    return stepBuilderFactory.get("Step1")
            .<DTOxyz, DTOxyz> chunk(100)
            .reader(dtoXyzItemReader(NULL because WILL_BE_INJECTED))
            .processor(XyzProcessor())
            .writer(writer)
            .build();        
}