我有一个合并到声明中:
private static final String UPSERT_STATEMENT = "MERGE INTO " + TABLE_NAME + " tbl1 " +
"USING (SELECT ? as KEY,? as DATA,? as LAST_MODIFIED_DATE FROM dual) tbl2 " +
"ON (tbl1.KEY= tbl2.KEY) " +
"WHEN MATCHED THEN UPDATE SET DATA = tbl2.DATA, LAST_MODIFIED_DATE = tbl2.LAST_MODIFIED_DATE " +
"WHEN NOT MATCHED THEN " +
"INSERT (DETAILS,KEY, DATA, CREATION_DATE, LAST_MODIFIED_DATE) " +
"VALUES (SEQ.NEXTVAL,tbl2.KEY, tbl2.DATA, tbl2.LAST_MODIFIED_DATE,tbl2.LAST_MODIFIED_DATE)";
这是执行方法:
public void mergeInto(final JavaRDD<Tuple2<Long, String>> rows) {
if (rows != null && !rows.isEmpty()) {
rows.foreachPartition((Iterator<Tuple2<Long, String>> iterator) -> {
JdbcTemplate jdbcTemplate = jdbcTemplateFactory.getJdbcTemplate();
LobCreator lobCreator = new DefaultLobHandler().getLobCreator();
while (iterator.hasNext()) {
Tuple2<Long, String> row = iterator.next();
String details = row._2();
Long key = row._1();
java.sql.Date lastModifiedDate = Date.valueOf(LocalDate.now());
Boolean isSuccess = jdbcTemplate.execute(UPSERT_STATEMENT, (PreparedStatementCallback<Boolean>) ps -> {
ps.setLong(1, key);
lobCreator.setBlobAsBytes(ps, 2, details.getBytes());
ps.setObject(3, lastModifiedDate);
return ps.execute();
});
System.out.println(row + "_" + isSuccess);
}
});
}
}
我需要在PLSQL内部插入多个此语句,如果可能,请填充10K的批量。
谢谢, 我
答案 0 :(得分:1)
最有效的方法是将您的数据批量加载到数据库中。与逐个上传(如您的示例)相比,我预计性能增益至少为1或2个数量级(“更大”的数据意味着通过批量插入获得的更少)。
您可以使用this answer中描述的技术先将记录批量插入临时表,然后使用临时表执行单个合并语句。 / p>