未调用Spring Batch跳过策略

时间：2020-09-25 17:17:19

标签： spring-boot spring-batch

我遇到的情况是，我的工作的第一步是对源数据库和目标数据库中的行数进行计数，因此我可以在工作执行过程中（可能需要几天的时间）展示一个实时进度监视器。

由于输入的特殊性，为工作提供数据的表的列表可能与两个数据库中实际上存在的表的列表不对应（我的输入可能具有数据库中不存在的表）。由于大多数列表应与数据库中存在的表匹配，因此我认为可以使用容错步骤来解决此问题。

在这个问题的背景下，我认为最重要的代码如下：

（创建步骤） 

@Bean( "partitionedSourceRowsCounterStep" )
public Step partitionedSourceRowsCounterStep( @Qualifier( "sourceRowCounterStep" ) Step rowCounterStep, 
                                          @Qualifier( "parallelTasksExecutor" ) TaskExecutor taskExecutor ) {

return stepBuilderFactory
            .get( "partitionedSourceRowsCounterStep" )
            .partitioner( "sourceRowCounterStep", new RowsCounterPartitioner( 
this.model.getEntityNames() ) )
            .step( rowCounterStep )
            .taskExecutor( taskExecutor )
            .build();
}

@Bean( "sourceRowCounterStep" )
public Step rowCounterStep( SourceRowsCounterReader reader, 
                        SourceRowsCounterInMemoryWriter writer,  
                        @Qualifier( "parallelTasksExecutor" ) TaskExecutor taskExecutor ) {

return stepBuilderFactory
            .get( "sourceRowCounterStep" )
            .<Map<String, Object>, Map<String, Object>>chunk( this.config.getChunkSize() )
            .reader( reader )
            .writer( writer )
            .faultTolerant()
            .skipPolicy( new SkipPolicy() {
                
                @Override
                public boolean shouldSkip( Throwable t, int skipCount ) throws SkipLimitExceededException {
                    System.out.println( "SKIP_POLICY WAS CALLED" );
                    return false;
                }
                
            } )
            .taskExecutor( taskExecutor )
            .throttleLimit( 1 )
            .build();
}

（行计数器阅读器） 

@StepScope
@Slf4j
@Component
public class SourceRowsCounterReader extends AbstractCursorItemReader<Map<String, Object>> {


@Override
protected void openCursor( Connection con ) {
    try {
        String sql = getSql();
        
        log.debug( "Query string: '" + sql + "'" );
        
        if( isUseSharedExtendedConnection() ) {
            preparedStatement = con.prepareStatement( sql, ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY, ResultSet.HOLD_CURSORS_OVER_COMMIT );
        } else {
            preparedStatement = con.prepareStatement( sql, ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY );
        }
        
        applyStatementSettings( preparedStatement );
        if( this.preparedStatementSetter != null ) {
            preparedStatementSetter.setValues( preparedStatement );
        }
        
        this.rs = preparedStatement.executeQuery(); // Exception happens here!
        
        handleWarnings( preparedStatement );
    } catch( SQLException se ) {
        close();
        throw getExceptionTranslator().translate("Executing query", getSql(), se);
    }

}

@Override
protected SQLExceptionTranslator getExceptionTranslator() {
    return this.customExceptionTranslator;
}

}

但是，当代码运行时，我会收到错误消息：

org.springframework.batch.item.ItemStreamException: Failed to initialize the reader
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:153) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader$$FastClassBySpringCGLIB$$ebb633d0.invoke(<generated>) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.2.9.RELEASE.jar:5.2.9.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) ~[spring-aop-5.2.9.RELEASE.jar:5.2.9.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.2.9.RELEASE.jar:5.2.9.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) ~[spring-aop-5.2.9.RELEASE.jar:5.2.9.RELEASE]
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136) ~[spring-aop-5.2.9.RELEASE.jar:5.2.9.RELEASE]
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124) ~[spring-aop-5.2.9.RELEASE.jar:5.2.9.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.9.RELEASE.jar:5.2.9.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) ~[spring-aop-5.2.9.RELEASE.jar:5.2.9.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) ~[spring-aop-5.2.9.RELEASE.jar:5.2.9.RELEASE]
at com.company.steps.SourceRowsCounterReader$$EnhancerBySpringCGLIB$$2dbb98e7.open(<generated>) ~[classes/:na]
at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:103) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.step.item.ChunkMonitor.open(ChunkMonitor.java:114) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:103) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep.open(TaskletStep.java:311) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:205) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.partition.support.TaskExecutorPartitionHandler$1.call(TaskExecutorPartitionHandler.java:138) [spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.partition.support.TaskExecutorPartitionHandler$1.call(TaskExecutorPartitionHandler.java:135) [spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_231]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_231]
Caused by: com.company.steps.jdbc.exception.TableOrViewDoesnExistException: Table doesn't exist; nested exception is java.sql.SQLSyntaxErrorException: ORA-00942: table or view does not exist

at com.company.steps.jdbc.oracle.OracleExceptionTranslator.translate(OracleExceptionTranslator.java:18) ~[classes/:na]
at com.company.steps.SourceRowsCounterReader.openCursor(SourceRowsCounterReader.java:99) ~[classes/:na]
at org.springframework.batch.item.database.AbstractCursorItemReader.doOpen(AbstractCursorItemReader.java:428) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:150) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
... 20 common frames omitted
Caused by: java.sql.SQLSyntaxErrorException: ORA-00942: table or view does not exist

此外，终端上没有打印“ SKIP_POLICY WAS CALLED”，即使由于先前的异常我也打了十二次。

根据docs异常应该被处理，而不管它们抛出的步骤链中的任何点。

任何人都可以在这里看到任何配置错误吗？

Spring Boot v2.3.4.RELEASE Spring Batch v4.2.4.RELEASE

1 个答案:

答案 0 :(得分：0)

根据您的堆栈跟踪记录，初始化读取器（SourceRowsCounterReader.openCursor）时会发生异常，这意味着该步骤目前尚未开始。

步骤正确后（即正确初始化了读取器/写入器），将调用跳过策略。

相关问题
最新问题