Question

我正在使用批处理分区来根据数据库表中某个列中的某个范围来获取和处理记录。在作业配置中的几个步骤之后出现分区步骤 stagingPartitionStep ，如下所示：

    <batch:job id="myBatchJob">
     <batch:step id="mainStep">
        <batch:tasklet task-executor="myTaskExecutor" transaction-manager="batchTransactionManager">
            <batch:chunk reader="myDataReader"
                processor="MyDataProcessor" writer="MyDataWriter" commit-interval="50">
            </batch:chunk>
            <batch:listeners>
                <batch:listener ref="myItemListener" />
            </batch:listeners>
        </batch:tasklet>
        <batch:next on="COMPLETED" to="checkConditionStep" />
    </batch:step>
    <batch:step id="checkConditionStep">
        <batch:tasklet task-executor="checkConditionExecutor"
            transaction-manager="batchTransactionManager" ref="checkConditionTasklet">
        </batch:tasklet>
        <batch:next on="FAILED" to="updateStagingTableStep" />
        <batch:next on="COMPLETED" to="stagingPartitionStep" />
    </batch:step>
    <batch:step id="updateStagingTableStep">
        <batch:tasklet task-executor="checkConditionExecutor"
            transaction-manager="batchTransactionManager" ref="updateStagingTasklet">
        </batch:tasklet>
    </batch:step>   
    <batch:step id="stagingPartitionStep">
        <batch:partition step="processStagingStep" partitioner="stagingProcessPartitioner">
            <batch:handler grid-size="10" task-executor="stagingProcessTaskExecutor" />
        </batch:partition>
    </batch:step>       
<batch:job>

分区程序和分区步骤：

<bean id="stagingProcessPartitioner"
        class="com.mycom.batch.partitioner.StagingProcessPartitioner"
        scope="step"> 
</bean>

<batch:step id="processStagingStep">
    <batch:tasklet transaction-manager="batchTransactionManager">
            <batch:chunk reader="stagingProcessorDataReader" writer="stagingProcessorDataWriter"
                commit-interval="50">
            </batch:chunk>
    </batch:tasklet>    
</batch:step>

任务执行者：

<bean id="myTaskExecutor"
    class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
    <property name="corePoolSize" value="20" />
    <property name="maxPoolSize" value="20" />
</bean>

<bean id="stagingProcessTaskExecutor"
    class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
    <property name="corePoolSize" value="10" />
    <property name="maxPoolSize" value="10" />
    <property name="allowCoreThreadTimeOut" value="true" />
</bean> 

<bean id="checkConditionStep"
    class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
    <property name="corePoolSize" value="1" />
    <property name="maxPoolSize" value="1" />
</bean>

分区程序实现：分区程序为每个分区步骤创建ExecutionContext，并放置一个前缀值，该值将在JDBC查询中用于根据凭证编号创建分区。

public class StagingProcessPartitioner implements Partitioner{

 public Map<String, ExecutionContext> partition(int gridSize) {
    Map<String, ExecutionContext> partitionMap = new HashMap<String, ExecutionContext>();
    for(int threadId = 0; threadId < gridSize; threadId++){
        ExecutionContext context = new ExecutionContext();
        String stepName = "step" + threadId;
        context.put("voucherSuffix", threadId);
        partitionMap.put(stepName, context);
        LOGGER.info("Created ExecutionContext for partioned step : " + stepName);
    }
    return partitionMap;
}}

数据阅读器：步骤上下文中的voucherSuffix用于JDBC查询以创建数据分区。因此，应在结尾为0,1,2 ......，9的凭证号码上创建10个部分。

    <bean id="stagingProcessorDataReader"
    class="org.springframework.batch.item.database.JdbcPagingItemReader" 
    scope="step">
    <property name="dataSource" ref="dataSource" />
    <property name="queryProvider" ref="stagingDataQueryProvider" />
    <property name="parameterValues">
        <map>
            <entry key="department" value="#{jobParameters[department]}" />
            <entry key="joiningDate" value="#{jobParameters[joiningDate]}" />
            <entry key="voucherSuffix" value="#{stepExecutionContext[voucherSuffix]}" />
        </map>
    </property> 
    <property name="pageSize" value="1000" />
    <property name="rowMapper" ref="myDataRowMapper"/>      
</bean>

<bean id="stagingDataQueryProvider"
    class="org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean">
    <property name="dataSource" ref="dataSource" />
    <property name="selectClause" value="SELECT EMP_ID, EMP_NAME, DOB, ADDRESS1, ADDRESS2" />           
    <property name="fromClause" value="EMP_STG_TBL" />
    <property name="whereClause" value="WHERE DEPT_ID=:department AND DOJ=:joiningDate AND VCHR LIKE '%:voucherSuffix'" />
    <property name="sortKeys">
        <map>
            <entry key="EMP_ID" value="ASCENDING"></entry>
        </map>
    </property>
</bean>

问题是，当作业执行完毕后，每个步骤都会被执行到分开步骤。分区器创建可以从日志语句确认的执行上下文，但不执行步骤processStagingStep，并且作业以状态COMPLETED结束。这个工作和分区步骤配置是否正确？

以下是日志声明

2015-02-23 03:03:04 INFO myTaskScheduler-3 SimpleStepHandler：146 - 执行步骤：[checkConditionStep] 2015-02-23 03:03:04 INFO myTaskScheduler-3 SimpleStepHandler：146 - 执行步骤：[stagingPartitionStep] 2015-02-23 03:03:04 INFO myTaskScheduler-3 StagingProcessPartitioner：29 - 为分区步骤创建ExecutionContext：step0 2015-02-23 03:03:04 INFO myTaskScheduler-3 StagingProcessPartitioner：29 - 为分区步骤创建ExecutionContext：step1 2015-02-23 03:03:04 INFO myTaskScheduler-3 StagingProcessPartitioner：29 - 为分区步骤创建ExecutionContext：step2 2015-02-23 03:03:04 INFO myTaskScheduler-3 StagingProcessPartitioner：29 - 为分区步骤创建ExecutionContext：step3 2015-02-23 03:03:04 INFO myTaskScheduler-3 StagingProcessPartitioner：29 - 为分区步骤创建ExecutionContext：step4 2015-02-23 03:03:04 INFO myTaskScheduler-3 StagingProcessPartitioner：29 - 为分区步骤创建ExecutionContext：step5 2015-02-23 03:03:04 INFO myTaskScheduler-3 StagingProcessPartitioner：29 - 为分区步骤创建ExecutionContext：step6 2015-02-23 03:03:04 INFO myTaskScheduler-3 StagingProcessPartitioner：29 - 为分区步骤创建ExecutionContext：step7 2015-02-23 03:03:04 INFO myTaskScheduler-3 StagingProcessPartitioner：29 - 为分区步骤创建ExecutionContext：step8 2015-02-23 03:03:04 INFO myTaskScheduler-3 StagingProcessPartitioner：29 - 为分区步骤创建ExecutionContext：step9 2015-02-23 03:03:05 INFO myTaskScheduler-3 SimpleJobLauncher：136 - 作业：[FlowJob：[name = myBatchJob]]使用以下参数完成：[]和以下状态：[COMPLETED]

Spring Batch Partitioning - 分区步骤不执行

0 个答案: