我的代码应该从表A中获取where子句,并将它们逐个应用于表REPORTS的记录,并将每个结果存储在表OUTPUTREPORT中。表A具有427条记录,而REPORTS具有约3900000条记录,大约需要4个小时。
这是我的工作,步骤和阅读器代码:
@Bean
public Job QueryJob() throws ClassNotFoundException, SQLException {
Connection dbConnection = dataSource.getConnection();
String selectSQL = "select WHERE_CONDITION from A";
PreparedStatement preparedStatement = dbConnection.prepareStatement(selectSQL);
ResultSet rs = preparedStatement.executeQuery();
List<Step> steps = new ArrayList<>();
while (rs.next()) {
Step step = createStep(rs.getString("WHERE_CONDITION"));
steps.add(step);
}
rs.close();
preparedStatement.close();
dbConnection.close();
return jobBuilderFactory.get("QueryJob")
.start(createParallelFlow(steps)).end().build();
}
int i = 1;
private Step createStep(String query) {
return stepBuilderFactory.get("convertStepFor" + i++)
.<Report, Report> chunk(5000).reader(createQueryReader(query))
.writer(writer()).build();
}
private Flow createParallelFlow(List<Step> steps) {
SimpleAsyncTaskExecutor taskExecutor = new SimpleAsyncTaskExecutor();
taskExecutor.setConcurrencyLimit(maxThreads);
List<Flow> flows = new ArrayList<>();
Iterator<Step> i = steps.iterator();
while (i.hasNext()) {
Step x = i.next();
Flow flow = new FlowBuilder<Flow>("flow_" + x.getName()).start(x)
.build();
if (flow != null)
flows.add(flow);
}
return new FlowBuilder<SimpleFlow>("parallelStepsFlow")
.split(taskExecutor).add(flows.toArray(new Flow[flows.size()]))
.build();
}
public JdbcCursorItemReader<Report> createQueryReader(String query) {
JdbcCursorItemReader<Report> reader = new JdbcCursorItemReader<>();
reader.setDataSource(dataSource);
reader.setSql("select * from reports" + query);
reader.setRowMapper(new ReportRowMapper());
return reader;
}
我是Spring Batch的新手。如何提高性能?