我正在尝试使用hibernate对数据库进行大量写入/更新,从上传的csv中获取数据,该数据可以在60,000到100,000行之间变化。我正在使用批量更新,但即便如此,它需要相当长的时间,即20多分钟。有人有任何提示吗?我想避免编写sql,但这是唯一的方法吗?
当前代码
for (String row : rows) {
String[] rowData = row.split(",");
try {
long userID = Long.parseLong(rowData[0]);
int platformID = Integer.parseInt(rowData[1]);
if (platformID == 14 && userID > 0) {
MCTileScheduleEntity schedule = new MCTileScheduleEntity.Builder()
.tile(tile)
.userID(userID)
.platformID(platformID)
.startDate(startDate)
.endDate(endDate)
.build();
if (existUser.add(userID)){
schedules.add(schedule);
} else {
error += 1;
errMessage += "Duplicated data detected for user id "+ userID + "\n";
}
} else {
error += 1;
errMessage += "Malformed data detected for user id "+ userID + "\n";
}
} catch (NumberFormatException e) {
error += 1;
errMessage += "Non-numerical input for user id " + rowData[0] + "\n";
}
}
if (error == 0) {
//write to database in batches
hb.batchUpdate(schedules);
tile.setTargetCount(Long.valueOf(schedules.size()));
hb.update(tile);
}