Question

要求 - 从Database1上的不同视图中提取数据（简单选择）并使用Spring Batch插入Database2上的表。

注意：每个步骤都不依赖于其他步骤 - 因此可以并行运行，而不是顺序运行。

解决方案：使用Spring Batch的Split Slows http://docs.spring.io/spring-batch/reference/html/configureStep.html#split-flows

批量配置 -

           <batch:job id="hrwardaily-batch-ps-hrit" xmlns="http://www.springframework.org/schema/batch">
           <batch:split id="splitFlow1" task-executor="taskExecutor">

                          <batch:flow>
                          <!-- STEP 1: READ BusinessUnit DATA IN CHUNKS AND WRITE IT -->
                          <step id="readAndWriteBusinessUnitPsToHrwar" >
                                         <tasklet transaction-manager="transactionManager">
                                                        <chunk reader="psViewBusinessUnitReader" writer="hrwarBusinessUnitStagingWriter"
                                                                       commit-interval="${hrwardaily.batch.ps.hrit.commitinterval}" />
                                                        <batch:listeners>
                                                                       <batch:listener ref="listener" />
                                                        </batch:listeners>
                                         </tasklet>
                                         <batch:next on="*" to="readAndWriteCompanyPsToHrwar" />
                                         <batch:next on="FAILED" to="readAndWriteCompanyPsToHrwar" />                                         
                          </step>

                          <!-- STEP 2: READ Company DATA IN CHUNKS AND WRITE IT -->
                          <step id="readAndWriteCompanyPsToHrwar">
                                         <tasklet transaction-manager="transactionManager">
                                                        <chunk reader="psViewCompanyReader" writer="hrwarCompanyStagingWriter"
                                                                       commit-interval="${hrwardaily.batch.ps.hrit.commitinterval}" />
                                                        <batch:listeners>
                                                                       <batch:listener ref="listener" />
                                                        </batch:listeners>
                                         </tasklet>
                                         <batch:next on="*" to="readAndWriteCountryPsToHrwar" />
                                         <batch:next on="FAILED" to="readAndWriteCountryPsToHrwar" />
                          </step>

                          <!-- STEP 3: READ Country DATA IN CHUNKS AND WRITE IT -->
                          <step id="readAndWriteCountryPsToHrwar">
                                         <tasklet transaction-manager="transactionManager">
                                                        <chunk reader="psViewCountryReader" writer="hrwarCountryStagingWriter"
                                                                       commit-interval="${hrwardaily.batch.ps.hrit.commitinterval}" />
                                                        <batch:listeners>
                                                                       <batch:listener ref="listener" />
                                                        </batch:listeners>                                          
                                         </tasklet>
                                         <batch:next on="*" to="readAndWriteCurrencyPsToHrwar" />
                                         <batch:next on="FAILED" to="readAndWriteCurrencyPsToHrwar" />
                          </step>

                          <!-- STEP 4: READ Currency DATA IN CHUNKS AND WRITE IT -->
                          <step id="readAndWriteCurrencyPsToHrwar">
                                         <tasklet transaction-manager="transactionManager">
                                                        <chunk reader="psViewCurrencyReader" writer="hrwarCurrencyStagingWriter"
                                                                       commit-interval="${hrwardaily.batch.ps.hrit.commitinterval}" />
                                                        <batch:listeners>
                                                                       <batch:listener ref="listener" />
                                                        </batch:listeners>
                                         </tasklet>
                                         <!-- <batch:next on="*" to="readAndWriteDepartmentPsToHrwar" />
                                         <batch:next on="FAILED" to="readAndWriteDepartmentPsToHrwar" /> -->
                          </step>
                          </batch:flow>


                          </batch:split>

           </batch:job>

例外：

org.springframework.batch.core.step.AbstractStep $ FatalException：检测到致命故障在org.springframework.batch.core.step.tasklet.TaskletStep $ 2.doInChunkContext（TaskletStep.java:310）在org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration（StepContextRepeatCallback.java:76）在org.springframework.batch.repeat.support.RepeatTemplate.getNextResult（RepeatTemplate.java:367）在org.springframework.batch.repeat.support.RepeatTemplate.executeInternal（RepeatTemplate.java:215）在org.springframework.batch.repeat.support.RepeatTemplate.iterate（RepeatTemplate.java:143）在org.springframework.batch.core.step.tasklet.TaskletStep.doExecute（TaskletStep.java:242）在org.springframework.batch.core.step.AbstractStep.execute（AbstractStep.java:198）在org.springframework.batch.core.job.AbstractJob.handleStep（AbstractJob.java:348）在org.springframework.batch.core.job.flow.FlowJob.access $ 0（FlowJob.java:1）在org.springframework.batch.core.job.flow.FlowJob $ JobFlowExecutor.executeStep（FlowJob.java:135）在org.springframework.batch.core.job.flow.support.state.StepState.handle（StepState.java:60）在org.springframework.batch.core.job.flow.support.SimpleFlow.resume（SimpleFlow.java:144）在org.springframework.batch.core.job.flow.support.SimpleFlow.start（SimpleFlow.java:124）在org.springframework.batch.core.job.flow.support.state.SplitState $ 1.call（SplitState.java:83）在org.springframework.batch.core.job.flow.support.state.SplitState $ 1.call（SplitState.java:1） at java.util.concurrent.FutureTask $ Sync.innerRun（FutureTask.java:303）在java.util.concurrent.FutureTask.run（FutureTask.java:138）在java.lang.Thread.run（Thread.java:662）引起：org.springframework.dao.OptimisticLockingFailureException：尝试使用错误的版本（2）更新步骤执行id = 3，其中当前版本为1 在org.springframework.batch.core.repository.dao.MapStepExecutionDao.updateStepExecution（MapStepExecutionDao.java:86）在org.springframework.batch.core.repository.support.SimpleJobRepository.update（SimpleJobRepository.java:167） at sun.reflect.NativeMethodAccessorImpl.invoke0（Native Method） at sun.reflect.NativeMethodAccessorImpl.invoke（NativeMethodAccessorImpl.java:39） at sun.reflect.DelegatingMethodAccessorImpl.invoke（DelegatingMethodAccessorImpl.java:25）在java.lang.reflect.Method.invoke（Method.java:597）在org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection（AopUtils.java:318）在org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint（ReflectiveMethodInvocation.java:183）在org.springframework.aop.framework.ReflectiveMethodInvocation.proceed（ReflectiveMethodInvocation.java:150）在org.springframework.transaction.interceptor.TransactionInterceptor.invoke（TransactionInterceptor.java:110）在org.springframework.aop.framework.ReflectiveMethodInvocation.proceed（ReflectiveMethodInvocation.java:172）在org.springframework.aop.framework.JdkDynamicAopProxy.invoke（JdkDynamicAopProxy.java:202）在$ Proxy16.update（未知来源）在org.springframework.batch.core.step.tasklet.TaskletStep $ 2.doInChunkContext（TaskletStep.java:307） ......还有17个 [30.10.2014 11:20:35] [SimpleAsyncTaskExecutor-1]错误：AbstractStep.execute（） - 执行步骤时遇到错误

感谢有人可以帮助我/指导我解决这个问题。

Answer 1

作为MapJobRepositoryFactoryBean状态（http://docs.spring.io/spring-batch/apidocs/org/springframework/batch/core/repository/support/MapJobRepositoryFactoryBean.html）的文档，它不是线程安全的，因此不适合多线程作业（包括具有拆分的作业）。切换到内存数据库（如HSQLDB），以便在作业存储库中具有线程安全性。

弹簧批 - 并行处理 - 分流

1 个答案: