SpringBatch itemReader用于很多内存

Question

我从数据库中读取它有很多行，是的，但是＆＃34;我想读者会在步骤中释放内存。它需要超过1Gb的内存。怎么可以改善它？使用另一种读者？

...
<bean id="database1" class="org.springframework.batch.item.database.JdbcCursorItemReader">
    <property name="name" value="bd1" />
    <property name="dataSource" ref="dataSource1" />
    <property name="sql" value="select image from info where image like '%/Images/%' or image like '%/temp/%'" />
    <property name="rowMapper">
        <bean class="foo.ImagesRowMapper" />
    </property>
</bean>

<batch:step id="1Step" next="2Step">
    <batch:tasklet>
        <batch:chunk reader="database1" processor="processor" writer="writer" commit-interval="20"/>
    </batch:tasklet>
</batch:step>
...

的RowMapper

public class ImagesRowMapper implements RowMapper<String> {

    @Override
    public String mapRow(ResultSet rs, int rowNum) throws SQLException {
        return rs.getString("image");
    }

}

。处理器

@Override
public String process(String item) throws Exception {
    if(logger.isDebugEnabled()){
        logger.debug("process. processing {}", item);
    }
    if (item == ""){
        return null;
    }
    return relativePath(item); //relativePath only change the string an return it
}

作家
没有做任何评论

Answer 1

您可以利用弹簧批量并行处理：

您可以定义taskExecutor

<bean id="taskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
        <property name="corePoolSize" value="5" />
        <property name="maxPoolSize" value="5" />

在您的tasklet中使用此taskexecutor

 <batch:tasklet ref="whatever_your_tasklet_name" transaction-manager="transactionManager" task-executor="taskExecutor" />

Spring docs：Scalability