在春季批次

Question

我在使用spring批处理以二进制格式创建大型报表时有一个要求，该报表保存在数据库中。所有工作数据都不能直接写入文件，也不能写入JobExecutionContext以外的工作表。

我知道通常你只会写入作业执行上下文，但我对于如何用这么大的报告（可能有几百兆字节）来解决这个问题我感到很困惑。 / p>

目前，我的Writer实现依赖于一个聚合器类，它被注入一个bean，然后是一个具有注入聚合器的TaskLet，其中将完成的报告写入数据库

问题在于我无法将我的聚合器范围扩展到step上下文，因此如果两个作业同时运行，它们将写入同一聚合器。

这是我目前的实施

域类

public class DataChunk {
    private int pageNumber;
    private byte[] data;
}

作家

public class FooWriter implements ItemWriter<DataChunk> {

    private DataChunkAggregator dataChunkAggregator;

    public void write(List<? extends DataChunk> dataChunks) throws Exception {
        dataChunks.stream().forEach(chunk -> dataChunkAggregator.addChunk(chunk.getPageNumber(), chunk.getData()));
    }
}

聚合

public class FooAggregator {
    private Map<int, byte> pagedData; // Key sorted implementation

    public void addChunk(int pageNumber, byte[] data) {
        pagedData.put(pageNumber, data)
    }

    public byte[] aggregate() {
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        pagedData.values.stream().forEach(data -> baos.write(data));
        return baos.toByteArray();
    }
}

报告编写Tasklet

public class ReportWritingTasklet implements TaskLet {

    private ReportRepository reportRepository;
    private FooAggregator fooAggregator;

    public RepeatStatus execute(StepContribution contribution, ChunkContext context) {
        byte[] data = fooAggregator.aggregate();
        reportRepository.getOne(reportId).setDataBytes(data);
    }
}

上下文

<?xml version="1.0" encoding="UTF-8"?>
<beans>
    <bean id=fooWriter class="FooWriter" scope="step"
        p:fooAggregator-ref="fooAggregator"/>

    <bean id="fooAggregator" class="FooAggregator"/>

    <bean id="reportWritingTasklet" class="ReportWritingTasklet" scope="step"
        p:fooAggregator-ref="fooAggregator"/>

    <batch:job id="fooJob">
        <batch:step id="generateReport" next="assembleReport">
            <batch:chunk reader="fooReader" processor="fooProcessor" writer="fooWriter"/>
        </batch:step>
        <batch:step id="assembleReport">
            <batch:tasklet class="ReportWritingTasklet"/>
        </batch:step>
     </batch:job>
</beans>

如果我尝试使FooAggregator步骤作用，我会将以下异常作为根本原因

Caused by: java.lang.IllegalStateException: Cannot convert value of type    [com.sun.proxy.$Proxy98 implementing org.springframework.aop.scope.ScopedObject,java.io.Serializable,org.springframework.aop.framework.AopInfrastructureBean,org.springframework.aop.SpringProxy,org.springframework.aop.framework.Advised] to required type [FooAggregator] for property 'fooAggregator': no matching editors or conversion strategy found

这是因为您只能将某些事情限定在该步骤范围内。

如何将执行上下文用作数据块的接收器，请记住它们会有很多并且它们会非常大？

Answer 1

我设法解决了这个问题。这不是Spring批次，但它符合我的要求。

基本上，有太多数据可以进出上下文。解决方案是保持作者本身的状态，并通过StepExecutionListener将Step保存在TransactionCallback的末尾。

更新的Writer

public class FooWriter extends StepExecutionListenerSupport implements ItemWriter<DataChunk> {

    private String reportId;
    private Map<Integer, byte[]> byteArrayMap = new ConcurrentSkipListMap<>();

    private TransactionTemplate transactionTemplate;
    private ReportRepository reportRepository;

    @Override
    public synchronized void write(List<? extends CaseChunk> caseChunks) throws Exception {
        caseChunks.stream().forEach(chunk -> {
            byteArrayMap.put(chunk.getPageNumber(), chunk.getBytes());
        });
    }

    @Override
    public void beforeStep(StepExecution stepExecution) {
        // No-op
    }

    @Override
    public ExitStatus afterStep(StepExecution stepExecution) {
        StringBuilder sb = new StringBuilder();
        for (byte[] byteArrayOutputStream : byteArrayMap.values()) {
            sb.append(new String(Base64.decode(byteArrayOutputStream)));
        }
        String encodedReportData = new String(Base64.encode(sb.toString().getBytes()));

        TransactionCallback<Report> transactionCallback = transactionStatus -> {
            Report report = reportRepository.getOne(this.reportId);
            report.setReportData(encodedReportData);
            reportRepository.save(report);
            return report;
        };

        // TransactionTemplate throws its own declared TransactionException, rethrows encountered RuntimeExceptions
        // and also Errors. Any problem writing the date kills the job, so it's OK to catch Throwable here instead
        // of trying to
        try {
            transactionTemplate.execute(transactionCallback);
        } catch (Throwable t) {
            LOGGER.error("Error saving report data ID:[{}]", reportId);
            return ExitStatus.FAILED.addExitDescription(t);
        }
        return ExitStatus.COMPLETED;
    }

}