Spring Batch JobExecutionException - ConcurrencyFailureException

时间：2017-06-13 11:48:29

标签： java spring multithreading spring-batch hsqldb

我无法在DEV环境中重现此错误，但它在我们的暂存和QA环境中随机发生。它还没有发生在PROD中，但这可能只是抽奖的运气。由于我无法重现错误，我对如何解决它感到茫然。我从阅读错误转储的最佳猜测是，Spring Job Transaction表未正确处理应用程序的多线程特性。

代码从许多来源获取数据，进行一些转换，然后创建数据文件以发送到其他来源。没有必要在运行之间保存状态，出错时我们运行清理脚本，并重新启动进程。这是一个单一的工作＆＃34;但是有很多步骤，这些步骤使用FlowBuilder并行运行步骤。

重新运行该过程始终会清除它。昨天我在我的DEV环境中一遍又一遍地运行这个过程，并且无法让它一次性发生。

这是有问题的bean：

@Bean
public DataSource dataSource() {    
    EmbeddedDatabaseBuilder builder = new EmbeddedDatabaseBuilder();
    EmbeddedDatabase db = builder
        .setType(EmbeddedDatabaseType.HSQL)     
        .build();
    return db;
}

这是错误：

2017-06-13 07:16:18.543 ERROR 403 --- [           main] o.s.batch.core.job.AbstractJob           : Encountered fatal error executing job

org.springframework.batch.core.JobExecutionException: Flow execution ended unexpectedly
    at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:140) ~[spring-batch-core-3.0.6.RELEASE.jar!/:3.0.6.RELEASE]
    at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:306) ~[spring-batch-core-3.0.6.RELEASE.jar!/:3.0.6.RELEASE]
    at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135) [spring-batch-core-3.0.6.RELEASE.jar!/:3.0.6.RELEASE]
    at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:128) [spring-batch-core-3.0.6.RELEASE.jar!/:3.0.6.RELEASE]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_121]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_121]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_121]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_121]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302) [spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) [spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) [spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) [spring-batch-core-3.0.6.RELEASE.jar!/:3.0.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208) [spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at com.sun.proxy.$Proxy34.run(Unknown Source) [na:na]
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:215) [spring-boot-autoconfigure-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:232) [spring-boot-autoconfigure-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:124) [spring-boot-autoconfigure-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:118) [spring-boot-autoconfigure-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:804) [spring-boot-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]
    at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:788) [spring-boot-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]
    at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:775) [spring-boot-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]
    at org.springframework.boot.SpringApplication.doRun(SpringApplication.java:366) [spring-boot-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:305) [spring-boot-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1124) [spring-boot-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1113) [spring-boot-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]
    at com.dcsg.batch.Main.main(Main.java:146) [UniversalFeedGenerator.jar!/:VERSION 1.1]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_121]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_121]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_121]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_121]
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:53) [UniversalFeedGenerator.jar!/:VERSION 1.1]
    at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
Caused by: org.springframework.batch.core.job.flow.FlowExecutionException: Ended flow=caliaFlow at state=caliaFlow.Calia Writer with exception
    at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:178) ~[spring-batch-core-3.0.6.RELEASE.jar!/:3.0.6.RELEASE]
    at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) ~[spring-batch-core-3.0.6.RELEASE.jar!/:3.0.6.RELEASE]
    at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:93) ~[spring-batch-core-3.0.6.RELEASE.jar!/:3.0.6.RELEASE]
    at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:90) ~[spring-batch-core-3.0.6.RELEASE.jar!/:3.0.6.RELEASE]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_121]
    ... 1 common frames omitted
Caused by: org.springframework.dao.ConcurrencyFailureException: PreparedStatementCallback; SQL [INSERT INTO BATCH_STEP_EXECUTION_CONTEXT (SHORT_CONTEXT, SERIALIZED_CONTEXT, STEP_EXECUTION_ID) VALUES(?, ?, ?)]; transaction rollback: serialization failure; nested exception is java.sql.SQLTransactionRollbackException: transaction rollback: serialization failure
    at org.springframework.jdbc.support.SQLExceptionSubclassTranslator.doTranslate(SQLExceptionSubclassTranslator.java:71) ~[spring-jdbc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:73) ~[spring-jdbc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81) ~[spring-jdbc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:645) ~[spring-jdbc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:866) ~[spring-jdbc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:927) ~[spring-jdbc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.persistSerializedContext(JdbcExecutionContextDao.java:233) ~[spring-batch-core-3.0.6.RELEASE.jar!/:3.0.6.RELEASE]
    at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.saveExecutionContext(JdbcExecutionContextDao.java:187) ~[spring-batch-core-3.0.6.RELEASE.jar!/:3.0.6.RELEASE]
    at org.springframework.batch.core.repository.support.SimpleJobRepository.add(SimpleJobRepository.java:171) ~[spring-batch-core-3.0.6.RELEASE.jar!/:3.0.6.RELEASE]
    at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source) ~[na:na]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_121]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_121]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302) [spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) [spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) [spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) ~[spring-tx-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:281) ~[spring-tx-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) ~[spring-tx-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208) [spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at com.sun.proxy.$Proxy32.add(Unknown Source) ~[na:na]
    at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source) ~[na:na]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_121]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_121]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302) [spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) [spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) [spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) [spring-batch-core-3.0.6.RELEASE.jar!/:3.0.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208) [spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at com.sun.proxy.$Proxy32.add(Unknown Source) ~[na:na]
    at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:144) ~[spring-batch-core-3.0.6.RELEASE.jar!/:3.0.6.RELEASE]
    at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:64) ~[spring-batch-core-3.0.6.RELEASE.jar!/:3.0.6.RELEASE]
    at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:67) ~[spring-batch-core-3.0.6.RELEASE.jar!/:3.0.6.RELEASE]
    at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) ~[spring-batch-core-3.0.6.RELEASE.jar!/:3.0.6.RELEASE]
    ... 5 common frames omitted
Caused by: java.sql.SQLTransactionRollbackException: transaction rollback: serialization failure
    at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source) ~[hsqldb-2.3.3.jar!/:2.3.3]
    at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source) ~[hsqldb-2.3.3.jar!/:2.3.3]
    at org.hsqldb.jdbc.JDBCPreparedStatement.fetchResult(Unknown Source) ~[hsqldb-2.3.3.jar!/:2.3.3]
    at org.hsqldb.jdbc.JDBCPreparedStatement.executeUpdate(Unknown Source) ~[hsqldb-2.3.3.jar!/:2.3.3]
    at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:873) ~[spring-jdbc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:866) ~[spring-jdbc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:629) ~[spring-jdbc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
    ... 36 common frames omitted
Caused by: org.hsqldb.HsqlException: transaction rollback: serialization failure
    at org.hsqldb.error.Error.error(Unknown Source) ~[hsqldb-2.3.3.jar!/:2.3.3]
    at org.hsqldb.error.Error.error(Unknown Source) ~[hsqldb-2.3.3.jar!/:2.3.3]
    at org.hsqldb.Session.executeCompiledStatement(Unknown Source) ~[hsqldb-2.3.3.jar!/:2.3.3]
    at org.hsqldb.Session.execute(Unknown Source) ~[hsqldb-2.3.3.jar!/:2.3.3]
    ... 41 common frames omitted

1 个答案:

答案 0 :(得分：0)

将数据源更改为此

  @Bean
  public BasicDataSource dataSource() {
    BasicDataSource dataSource = new BasicDataSource();
    dataSource.setDriverClassName("org.hsqldb.jdbcDriver");
    dataSource.setUrl("jdbc:hsqldb:mem:testdb;sql.enforce_strict_size=true;hsqldb.tx=mvcc");
    dataSource.setUsername("sa");
    dataSource.setPassword("");
    return dataSource;
  }

修正了它。关键的变化是使用嵌入式数据源不支持的“tx = mvcc”。这使“多版本并发控制”

这个问题解释了MVCC是什么以及为什么在答案中使用它：

Database: What is Multiversion Concurrency Control (MVCC) and who supports it?

相关问题
最新问题