Question

我正在尝试使用ojdbc14.jar驱动程序和Spring的SimpleJdbcTemplate batchUpdate方法将超过100,000条记录插入到没有主键的Oracle 9i表中。这是我的代码片段：

private static final String TABLE_INSERT = "insert into TABLE_FINAL (ID, START_TIME, VALUE) VALUES (ID_SEQ.NEXTVAL, :startTime, :value)";

log.info("inputData list size={}",inputData.size());
Object[] dataArray = inputData.toArray();
log.info("dataArray length={}",dataArray.length);

final SqlParameterSource[] batch = SqlParameterSourceUtils.createBatch(inputData.toArray());
log.info("SqlParamterSource length={}", batch.length);

final int[] inserted = getJdbcTemplateJoa().batchUpdate(TABLE_INSERT, batch);

for(int i=0; i < inserted.length; i++){
if(inserted[i] != -2){
    System.out.println("i="+i +" insert[i]="+inserted[i]);
    System.out.println(batch[i]);
}

}

inputData List，dataArray和批处理长度的大小都是相同的期望值。 batchUpdate完成时不会抛出任何异常，后续for循环不会打印任何内容，因为inserted数组中的每个项都返回-2（成功）。但是，目标表中只保留了42,000条记录，而不是预期的100,000条记录。

如果我将batchUpdate替换为对输入集合进行循环并对每个项目执行更新，则会保留100,000多条记录。但是，我想使用batchUpdate来利用改进的性能。

有没有人知道为什么batchUpdate不起作用？我不禁想到它与丢失的主键有关。

以下是源表中用于填充inputData List的数据：

0.1933,-0.0253,0,0,4/16/2011 5:00:00 AM,4/16/2011 6:00:00 AM,12,9,1,1
0.1917,-0.0253,0,0,4/16/2011 6:00:00 AM,4/16/2011 7:00:00 AM,12,9,1,1
0.1936,-0.0253,0,0,4/16/2011 7:00:00 AM,4/16/2011 8:00:00 AM,12,9,1,1
0.2017,-0.0253,0,0,4/16/2011 8:00:00 AM,4/16/2011 9:00:00 AM,12,9,1,1
0.2083,-0.0253,0,0,4/16/2011 9:00:00 AM,4/16/2011 10:00:00 AM,12,9,1,1
0.2133,-0.0253,0,0,4/16/2011 10:00:00 AM,4/16/2011 11:00:00 AM,12,9,1,1
0.2238,-0.0253,0,0,4/16/2011 11:00:00 AM,4/16/2011 12:00:00 PM,12,9,1,1
0.2309,-0.0253,0,0,4/16/2011 12:00:00 PM,4/16/2011 1:00:00 PM,12,9,1,1
0.2319,-0.0253,0,0,4/16/2011 1:00:00 PM,4/16/2011 2:00:00 PM,12,9,1,1
0.231,-0.0253,0,0,4/16/2011 2:00:00 PM,4/16/2011 3:00:00 PM,12,9,1,1
0.2283,-0.0253,0,0,4/16/2011 3:00:00 PM,4/16/2011 4:00:00 PM,12,9,1,1
0.2216,-0.0253,0,0,4/16/2011 4:00:00 PM,4/16/2011 5:00:00 PM,12,9,1,1
0.2164,-0.0253,0,0,4/16/2011 5:00:00 PM,4/16/2011 6:00:00 PM,12,9,1,1
0.2155,-0.0253,0,0,4/16/2011 6:00:00 PM,4/16/2011 7:00:00 PM,12,9,1,1
0.2162,-0.0253,0,0,4/16/2011 7:00:00 PM,4/16/2011 8:00:00 PM,12,9,1,1
0.2187,-0.0253,0,0,4/16/2011 8:00:00 PM,4/16/2011 9:00:00 PM,12,9,1,1
0.2203,-0.0253,0,0,4/16/2011 9:00:00 PM,4/16/2011 10:00:00 PM,12,9,1,1
0.2296,-0.0253,0,0,4/16/2011 10:00:00 PM,4/16/2011 11:00:00 PM,12,9,1,1
0.2323,-0.0253,0,0,4/16/2011 11:00:00 PM,4/17/2011,12,9,1,1
0.2293,-0.0253,0,0,4/17/2011,4/17/2011 1:00:00 AM,12,9,1,1
0.2154,-0.0253,0,0,4/17/2011 1:00:00 AM,4/17/2011 2:00:00 AM,12,9,1,1
0.2088,-0.0253,0,0,4/17/2011 2:00:00 AM,4/17/2011 3:00:00 AM,12,9,1,1
0.202,-0.0253,0,0,4/17/2011 3:00:00 AM,4/17/2011 4:00:00 AM,12,9,1,1
0.1916,-0.0253,0,0,4/17/2011 4:00:00 AM,4/17/2011 5:00:00 AM,12,9,1,1

以下是batchUpdate后持久化的内容：

47987296,4/19/2011 4:37:15 PM,0.1933,-0.0253,4/16/2011 5:00:00 AM,4/16/2011 6:00:00 AM,4/19/2011 4:37:28 PM,9,12,1,1,04-15-2011 
47961249,4/19/2011 4:37:15 PM,0.2238,-0.0253,4/16/2011 11:00:00 AM,4/16/2011 12:00:00 PM,4/19/2011 4:37:28 PM,9,12,1,1,04-15-2011 
47966094,4/19/2011 4:37:15 PM,0.2309,-0.0253,4/16/2011 12:00:00 PM,4/16/2011 1:00:00 PM,4/19/2011 4:37:28 PM,9,12,1,1,04-15-2011 
47968596,4/19/2011 4:37:15 PM,0.2319,-0.0253,4/16/2011 1:00:00 PM,4/16/2011 2:00:00 PM,4/19/2011 4:37:28 PM,9,12,1,1,04-15-2011 
47972962,4/19/2011 4:37:15 PM,0.231,-0.0253,4/16/2011 2:00:00 PM,4/16/2011 3:00:00 PM,4/19/2011 4:37:28 PM,9,12,1,1,04-15-2011 
47978129,4/19/2011 4:37:15 PM,0.2283,-0.0253,4/16/2011 3:00:00 PM,4/16/2011 4:00:00 PM,4/19/2011 4:37:28 PM,9,12,1,1,04-15-2011 
47982943,4/19/2011 4:37:15 PM,0.2216,-0.0253,4/16/2011 4:00:00 PM,4/16/2011 5:00:00 PM,4/19/2011 4:37:28 PM,9,12,1,1,04-15-2011 
48005719,4/19/2011 4:37:15 PM,0.2164,-0.0253,4/16/2011 5:00:00 PM,4/16/2011 6:00:00 PM,4/19/2011 4:37:28 PM,9,12,1,1,04-15-2011 
47990490,4/19/2011 4:37:15 PM,0.2088,-0.0253,4/17/2011 2:00:00 AM,4/17/2011 3:00:00 AM,4/19/2011 4:37:28 PM,9,12,1,1,04-15-2011 
47993531,4/19/2011 4:37:15 PM,0.202,-0.0253,4/17/2011 3:00:00 AM,4/17/2011 4:00:00 AM,4/19/2011 4:37:28 PM,9,12,1,1,04-15-2011 
48000722,4/19/2011 4:37:15 PM,0.1916,-0.0253,4/17/2011 4:00:00 AM,4/17/2011 5:00:00 AM,4/19/2011 4:37:28 PM,9,12,1,1,04-15-2011

源表中的24行在目标表中也应该有24行，但只填充了11行。

Answer 1

当使用带有ojdbc14.jar的SimpleJdbcTemplate.batchUpdate（String sql，SqlParameterSource [] source）和大量数据（超过60K）时，我在原始帖子中描述的目标表中缺少数据。我发现如果我将输入数据分成10K块，数据就会成功保存。我也尝试使用正确持久化的JdbcTemplate.batchUpdate（String [] sql）方法，但比循环和调用SimpleJdbcTemplate.update慢。从好的方面来说，JdbcTemplate.batchUpdate（String [] sql）返回一个int []，其中数组中的每个项目都包含受影响的行数。

我将我的Oracle驱动程序更改为ojdbc6.jar，并使用SimpleJdbcTemplate.batchUpdate（String sql，SqlParamterSource [] source）重新测试，传递了所有100,000多个源记录，并且工作正常！不幸的是，我们有其他需要ojdbc14.jar的依赖项，所以我们还无法升级。

对于最终解决方案，数据将被分解为10K块，如下所示，并且将在batchUpdate之后添加验证数据被持久化的SQL查询。

if(inputData.size() > 10000){

            int beginIndex =0;
            int endIndex = 10000;
            List<InputData> partialList = null;
            while(beginIndex < inputData.size()){
                partialList = inputData.subList(beginIndex, endIndex);

                final SqlParameterSource[] batch = SqlParameterSourceUtils.createBatch(partialList.toArray());

                getJdbcTemplateJoa().batchUpdate(TABLE_INSERT, batch);

                beginIndex = endIndex;
                endIndex = endIndex + 10000 < inputData.size() ? endIndex + 10000 : inputData.size();
            }
} else{

            final SqlParameterSource[] batch = SqlParameterSourceUtils.createBatch(inputData.toArray());
            getJdbcTemplateJoa().batchUpdate(TABLE_INSERT, batch);

        }

Answer 2

也许存在被捕获但未通过的例外情况。尝试安装servererror触发器，以确定Oracle是否向客户端传递了任何异常。

Here你会找到一个例子。

顺便说一句，我对你在工作中取得的性能提升感兴趣。如果不起作用我不会感到惊讶......

Oracle批量插入没有主键缺失插入

2 个答案: