春季批ItemWriter异常正在杀死工作

时间：2020-10-28 12:41:57

标签： postgresql spring-boot jpa spring-batch spring-transactions

我有一个Spring Batch应用程序，它将通过rest API读取数据并将数据写入DB。

下面是我的代码。

配置：

@Bean
@Qualifier("batchJob")
public Job batchJob() {
    return jobBuilderFactory.get("batchJob").incrementer(new RunIdIncrementer()).listener(batchJobExecutionListener)
            .start(firstStep)
            .next(secondStep)
            .build();
}

@Bean
public Step firstStep() {
    return stepBuilderFactory.get("firstStep").<Input, Output>chunk(AppConst.BATCH_CHUNK)
            .reader(firstReader)
            .processor(firstProcessor)
            .writer(firstWriter)
            .faultTolerant()
            .skip(Exception.class)
            .skipLimit(99999)
            .listener(new FirstSkipListener())
            .build();
}

FirstSkipListener.class：

@Component
@Slf4j
public class FirstSkipListener implements SkipListener<Input, Output> {

    @Override
    public void onSkipInProcess(Input arg0, Throwable arg1) {
        // Do nothing in onSkipInProcess
    }

    @Override
    public void onSkipInRead(Throwable arg0) {
        // Do nothing in onSkipInRead
    }

    @Override
    public void onSkipInWrite(Output output, Throwable arg1) {
        log.error("Skipped output = {} ", output);
    }

}

FirstWriter.class：

@Component
@Slf4j
public class FirstWriter implements ItemWriter<Output> {

    @Autowired
    private OutputRepository outputRepository;
    
    @Autowired
    private BatchRunRepository batchRunRepository;
    
    private Integer batchId;
    
    private Integer updateCount;
    
    /**
     * This method will write output list in db.
     * @param outputs
     * @return void
     * @throws Exception
     */
    @Override
    public void write(List<? extends Output> outputs) {
        log.info("Start writing output data.");
        if(!CollectionUtils.isEmpty(outputs)) {
            try {
                updateCount = outputs.size();
                outputRepository.saveAll(outputs);
                outputRepository.flush();
                log.info("End writing output data.");
            } catch (Exception e) {
                log.error("Error occured while writing output data: {}", e.getMessage());
            }
        } else {
            log.info("No data to be writtern for output.");
        }
    }
}

OutputRepository.class：

@Repository
public interface OutputRepository extends JpaRepository<Output, Long> {
}

application.yml：

spring:
  jpa:
    database-platform: org.hibernate.dialect.PostgreSQLDialect
    properties:
      hibernate:
        dialect: org.hibernate.dialect.PostgreSQLDialect
        default_schema: my_schema
        format_sql: true
        jdbc: 
          lob:
            non_contextual_creation: true
  batch:
    initialize-schema: always
    initializer:
      enabled: false
    job:
      enabled: false
  main:
    allow-bean-definition-overriding: true
  jackson:
    serialization:
      indent_output: true
  profiles: local
    
  datasource:
    url: jdbc:postgresql://localhost:5432/mydb?currentSchema=my_schema
    username: postgres
    password: 12345
    platform: postgres
  jpa:             
    show-sql: false
    hibernate:
      ddl-auto: validate

现在，当我开始工作时，我正处于例外之下。

2020-10-28 17:56:01.520  INFO [my-app,5764b6867c680cf1,5764b6867c680cf1,false] 17548 --- [nio-8080-exec-1] c.m.c.d.m.b.job.writer.FirstWriter     : Start writing output data.
2020-10-28 17:56:02.108 ERROR [my-app,5764b6867c680cf1,5764b6867c680cf1,false] 17548 --- [nio-8080-exec-1] o.h.engine.jdbc.spi.SqlExceptionHelper   : ERROR: null value in column "output_name" violates not-null constraint
  Detail: Failing row contains (486, null, 4, null, null, null, null, Need Description, data, null, 439e00ef-45d5-3a4b-ab65-f620f641b6d3, f, A, user, 2020-10-28 17:56:02.101, 2020-10-28 17:56:02.101, user, null, ).
2020-10-28 17:56:02.118 ERROR [my-app,5764b6867c680cf1,5764b6867c680cf1,false] 17548 --- [nio-8080-exec-1] c.m.c.d.m.b.job.writer.FirstWriter     : Error occured while writing output data: could not execute statement; SQL [n/a]; constraint [output_name]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement
2020-10-28 17:56:02.250 ERROR [my-app,5764b6867c680cf1,5764b6867c680cf1,false] 17548 --- [nio-8080-exec-1] o.s.batch.core.step.tasklet.TaskletStep  : JobRepository failure forcing rollback

org.springframework.jdbc.UncategorizedSQLException: PreparedStatementCallback; uncategorized SQLException for SQL [UPDATE BATCH_STEP_EXECUTION_CONTEXT SET SHORT_CONTEXT = ?, SERIALIZED_CONTEXT = ? WHERE STEP_EXECUTION_ID = ?]; SQL state [25P02]; error code [0]; ERROR: current transaction is aborted, commands ignored until end of transaction block; nested exception is org.postgresql.util.PSQLException: ERROR: current transaction is aborted, commands ignored until end of transaction block
    at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:89) ~[spring-jdbc-5.2.6.RELEASE.jar:5.2.6.RELEASE]
    at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81) ~[spring-jdbc-5.2.6.RELEASE.jar:5.2.6.RELEASE]
    at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81) ~[spring-jdbc-5.2.6.RELEASE.jar:5.2.6.RELEASE]
    at org.springframework.jdbc.core.JdbcTemplate.translateException(JdbcTemplate.java:1443) ~[spring-jdbc-5.2.6.RELEASE.jar:5.2.6.RELEASE]
    at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:633) ~[spring-jdbc-5.2.6.RELEASE.jar:5.2.6.RELEASE]
    at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:862) ~[spring-jdbc-5.2.6.RELEASE.jar:5.2.6.RELEASE]
    at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:917) ~[spring-jdbc-5.2.6.RELEASE.jar:5.2.6.RELEASE]
    at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.persistSerializedContext(JdbcExecutionContextDao.java:236) ~[spring-batch-core-4.2.2.RELEASE.jar:4.2.2.RELEASE]
    at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.updateExecutionContext(JdbcExecutionContextDao.java:163) ~[spring-batch-core-4.2.2.RELEASE.jar:4.2.2.RELEASE]
    at org.springframework.batch.core.repository.support.SimpleJobRepository.updateExecutionContext(SimpleJobRepository.java:210) ~[spring-batch-core-4.2.2.RELEASE.jar:4.2.2.RELEASE]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_261]
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_261]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_261]
    at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_261]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) [spring-aop-5.2.6.RELEASE.jar:5.2.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) [spring-aop-5.2.6.RELEASE.jar:5.2.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) [spring-aop-5.2.6.RELEASE.jar:5.2.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:366) ~[spring-tx-5.2.6.RELEASE.jar:5.2.6.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118) ~[spring-tx-5.2.6.RELEASE.jar:5.2.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) [spring-aop-5.2.6.RELEASE.jar:5.2.6.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) [spring-aop-5.2.6.RELEASE.jar:5.2.6.RELEASE]
    at com.sun.proxy.$Proxy158.updateExecutionContext(Unknown Source) ~[na:na]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_261]
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_261]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_261]
    at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_261]

在我的代码中，我正在设置跳过策略，并且期望如果我最初在输出列表中传递10个元素并且第5条记录出现问题，那么spring batch应该抛出异常并重试调用write方法10次。这样一来，只有一条记录将被跳过，而另外9条将保留在数据库中。

但就我而言，我的批处理应用程序完全停止了。甚至无法更新sprint维护的BATCH_STEP_EXECUTION_CONTEXT表。

请注意，我正在使用JPA存储库，而不是尝试在任何地方管理事务。甚至没有考虑考虑spring的事务管理器会代表我管理它。

1 个答案:

答案 0 :(得分：0)

好吧，我得到了错误。在我的写课中，我有一个catch块，用于打印和异常。我删除了try catch块并让spring批处理查找异常并调用跳过监听器。

相关问题
最新问题