Question

我有一份每天安排的工作。作业从API中批量读取数据，并将结果提交到Spring ConcurrentTaskExecutor线程池中。 ConcurrentTaskExecutor中定义的任务处理数据并将其写入DB。

考虑到并发执行，我的问题是如何获得合并矩阵，如成功，读者和处理器的失败

public class JobHandler{
   @Autowired
   Reader reader;
   //invoked once in a day
   public void execute(){
      reader.read();
     // Print the total read count and failures
     // Print the total process count and failures
   } 
}
@Component
public class Reader{
    @Autiwrired
    private ConcurrentTaskExecutor concurrentTaskExecutor;
    @Autiwrired
    private Processor processor;
    public void read(){

     while(true){
     //read the data from third partyAPI in batches
      try{
        processor.submit(()-> concurrentTaskExecutor.submit(processor.process()));
       if(//no results){
           break;
       }
      }catch(Exception e){
      } 
     }
  }
}
@Component
public class Processor{
    public void process(Data data){
     try{
       //read the data from third partyAPI in batches
      //do some operations and writes to DB
     }catch(Exception e){
     }
     }
  }
}

Answer 1

使用回调异步获取结果。 F.e。

public class JobHandler {
        @Autowired
        Reader reader;

        //invoked once in a day
        public void execute() {
            reader.read(
                    new ResultCallback() {

                        @Override
                        public void onRead(long success, long failures) {
                            // Print the total read count and failures
                        }

                        @Override
                        public void onProcess(long success, long failures) {
                            // Print the total process count and failures
                        }
                    }
            );
        }
    }

    public interface ResultCallback {
        void onRead(long success, long failures);
        void onProcess(long success, long failures);
    }

    @Component
    public class Reader {
        @Autiwrired
        private ConcurrentTaskExecutor concurrentTaskExecutor;
        @Autiwrired
        private Processor processor;

        public void read(ResultCallback callback) {

            while (true) {
                //read the data from third partyAPI in batches
                DataResult dataResult = ...;
                callback.onRead(dataResult.getSuccess(), dataResult.getFailure());
                Data data = dataResult.getData();
                try {
                    processor.submit(() -> concurrentTaskExecutor.submit(
                            (Runnable) () -> {
                                ProcessorResult processorResult = processor.process(data);
                                callback.onProcess(processorResult.getSuccess(), processorResult.getFailure());
                            }
                    ));
                    if (//no results){
                    break;
                }
            }catch(Exception e){
            }
        }
    }

    @Component
    public class Processor {
        public ProcessorResult process(Data data) {
            try {
                //read the data from third partyAPI in batches
                //do some operations and writes to DB
            } catch (Exception e) {
            }
        }
    }

Answer 2

如果时间紧迫-记录失败和成功信息，然后使用zgrep | wc -l解析日志（我假设您已经具有log4j之类的日志记录功能）。

给更多时间，用spring-batch重写它，并从spring-batch数据库表中获取统计信息。

Spring Java：如何获得成功的矩阵，在线程池中运行的任务失败

2 个答案: