无法在远程分区中执行并行执行

时间：2019-06-13 14:58:34

标签： java spring-batch

要求：-    使用spring batch定义远程分区

  

看起来所有分区都没有执行并行执行。   据我了解，例如，如果gridSize = 3，则所有三个分区都应与从属step（reader \ process \ writer）类并行执行。

    Configurations:-
        TaskExecutor:-
           corePoolSize = 3
           maxPoolSize  = 20
           queueCapacity = 100
       gridSize = 3
       chunkSize = 10

我已经定义了分区程序类，该类将返回与网格大小相同的executionContext。    ExecutionContext = {PART3 = start = 0，PART1 = start = 0，PART2 = start = 0} 

ReaderClass:-

  public void beforeStep(StepExecution stepExecution) {
    log.info(stepExecution);
  }

  public Scope read(){
     --- some operations---
      return new Scope();
  }

  ProcessClass:-

   public Data process(Scope scope){
      -- some operations --
     return new Data(scope);
   } 

   WriterClass :-

    public void write(List<? extends Data> dataItemsSize){

      -- some operations --
     }

日志：-

            [INFO ] 2019-06-13 09:18:13.389 [Container-28] Reader - beforeStep() reader: stepExecution=StepExecution: id=52175, version=1, name=MatchStep:PART2, status=STARTED, exitStatus=EXECUTING, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=0, rollbackCount=0, exitDescription=
        [INFO ] 2019-06-13 09:18:13.452 [Container-49] Reader - beforeStep() reader: stepExecution=StepExecution: id=52173, version=1, name=MatchStep:PART3, status=STARTED, exitStatus=EXECUTING, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=0, rollbackCount=0, exitDescription=
        [INFO ] 2019-06-13 09:18:13.455 [Container-28] Reader - read() - END: scope=Scope{partitionerId=PART2, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2855, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.455 [Container-28] Reader - read() - END: scope=Scope{partitionerId=PART2, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2841, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.456 [Container-28] Reader - read() - END: scope=Scope{partitionerId=PART2, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2851, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.456 [Container-28] Reader - read() - END: scope=Scope{partitionerId=PART2, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2863, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.476 [Container-30] Reader - beforeStep() reader: stepExecution=StepExecution: id=52171, version=1, name=MatchStep:PART1, status=STARTED, exitStatus=EXECUTING, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=0, rollbackCount=0, exitDescription=
        [INFO ] 2019-06-13 09:18:13.477 [Container-49] Reader - read() - END: scope=Scope{partitionerId=PART3, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2845, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.477 [Container-49] Reader - read() - END: scope=Scope{partitionerId=PART3, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2847, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.477 [Container-49] Reader - read() - END: scope=Scope{partitionerId=PART3, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2861, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.477 [Container-49] Reader - read() - END: scope=null
        [INFO ] 2019-06-13 09:18:13.483 [Container-49] Process - process(): scope=Scope{partitionerId=PART3, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2845, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.483 [Container-28] Process - process(): scope=Scope{partitionerId=PART2, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2855, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.506 [Container-30] Reader - read() - END: scope=Scope{partitionerId=PART1, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2857, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.506 [Container-30] Reader - read() - END: scope=Scope{partitionerId=PART1, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2843, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.506 [Container-30] Reader - read() - END: scope=Scope{partitionerId=PART1, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2849, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.506 [Container-30] Reader - read() - END: scope=Scope{partitionerId=PART1, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2853, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.506 [Container-30] Reader - read() - END: scope=Scope{partitionerId=PART1, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2859, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.506 [Container-30] Reader - read() - END: scope=Scope{partitionerId=PART1, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2865, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.506 [Container-30] Reader - read() - END: scope=null
        [INFO ] 2019-06-13 09:18:13.507 [Container-30] Process - process(): scope=Scope{partitionerId=PART1, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2857, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.509 [Container-28] Process - process(): scope=Scope{partitionerId=PART2, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2841, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.509 [Container-49] Process - process(): scope=Scope{partitionerId=PART3, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2847, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.513 [Container-28] Process - process(): scope=Scope{partitionerId=PART2, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2851, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.513 [Container-49] Process - process(): scope=Scope{partitionerId=PART3, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2861, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.516 [Container-28] Process - process(): scope=Scope{partitionerId=PART2, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2863, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.593 [Container-30] Process - process(): scope=Scope{partitionerId=PART1, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2843, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.598 [Container-30] Process - process(): scope=Scope{partitionerId=PART1, openDate=Thu Jun 13 09:18:13 CDT 2019, execScopeId=2849, execId=1321, iTC=CX}
        [INFO ] 2019-06-13 09:18:13.608 [Container-28] Writer - write(): dataItemsSize=4
        [INFO ] 2019-06-13 09:18:13.608 [Container-49] Writer - write(): dataItemsSize=3

1 个答案:

答案 0 :(得分：0)

看起来您正在使用TaskExecutorPartitionHandler，并根据您的日志，按预期由不同线程（Container-28，Container-30，Container-49）并行执行分区。

  

要求：-使用spring batch定义远程分区

如果要配置远程分区，则需要使用MessageChannelPartitionHandler。您可以找到有关如何使用它的示例here。

相关问题
最新问题