插入CLOB非常慢

时间:2013-02-05 10:24:40

标签: sql oracle plsql clob varchar2

与VARCHAR2相比,将'TinyString'字符串(只是一个示例字符串)插入到行内CLOB时,我遇到了显着的性能下降。我的理解是,当存储<在启用了STORAGE IN ROW的CLOB中有4000个字节,数据以与VARCHAR2相同的方式有效存储(除非它'溢出'4000bytes)并且应该没有显着的性能下降。但是,我的基准测试程序*表明,将相同的数据插入到CLOB中比插入VARCHAR2慢15倍

看看下面的代码:

我有很多桌子,每个桌子都附有一个类似下面的复合触发器:

CREATE OR REPLACE TRIGGER mdhl_basic_trigger_compound
  FOR INSERT OR UPDATE OR DELETE ON target_table

  COMPOUND TRIGGER TYPE EVENTS_HIST IS TABLE OF log_table%ROWTYPE INDEX BY PLS_INTEGER;
                                                coll_events_hist EVENTS_HIST;
                                                ctr PLS_INTEGER := 0;
                                                my_bgroup VARCHAR2(3);

  BEFORE EACH ROW IS    
    BEGIN

      IF INSERTING OR UPDATING THEN
        my_bgroup  := :NEW.BGROUP;
      ELSE
        my_bgroup  := :OLD.BGROUP;
      END IF;

      ctr := ctr + 1;
      coll_events_hist(ctr).BGROUP := my_bgroup;
      coll_events_hist(ctr).TABLE_NAME := 'BASIC_MDHL';
      coll_events_hist(ctr).EVENT_TS := current_timestamp;         
      coll_events_hist(ctr).EVENT_RAW := 'TinyString';

  END BEFORE EACH ROW;

  AFTER STATEMENT IS 
    BEGIN
      FORALL counter IN 1 .. coll_events_hist.count() 
           INSERT INTO log_table VALUES coll_events_hist(counter); 
  END AFTER STATEMENT; 
END mdhl_basic_trigger_compound;

target_table上执行任何操作后,上述触发器会将coll_events_hist类型中填充的数据存储到log_table,其定义方式如下:

CREATE TABLE "USERNAME"."LOG_TABLE" 
   (  "BGROUP" VARCHAR2(3) NOT NULL ENABLE, 
        "TABLE_NAME" VARCHAR2(255) NOT NULL ENABLE, 
      "EVENT_TS" TIMESTAMP (7) DEFAULT current_timestamp, 
      "EVENT_RAW" CLOB
   ) 
  SEGMENT CREATION IMMEDIATE 
  PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 NOCOMPRESS LOGGING
  STORAGE(INITIAL 65536 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645
  PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT FLASH_CACHE DEFAULT CELL_FLASH_CACHE DEFAULT)
  TABLESPACE "USERS" 
 LOB ("EVENT_RAW") STORE AS BASICFILE "EV_RAW_SEG"(
  TABLESPACE "USERS" ENABLE STORAGE IN ROW CHUNK 16384 PCTVERSION 5
  CACHE 
  STORAGE(INITIAL 65536 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645
  PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT FLASH_CACHE DEFAULT CELL_FLASH_CACHE DEFAULT))

我的设置是: Windows 7 SP1, Oracle 11g

*我的benchamrking程序迭代10次,在每次迭代中更新target_table上的21k行。

2 个答案:

答案 0 :(得分:3)

在你的情况下是“tinystring”总是< 32767?

你的时间将被浪费在FORALL部分,查找你所做的所有临时蠢事。

你会发现每个行部分的插入效果更好:

例如在我的测试系统上使用你的高音扳机:

SQL> insert into target_Table select 'ABC' from dual connect by level <= 10000;

10000 rows created.

Elapsed: 00:00:10.49

将触发器设为:

SQL> CREATE OR REPLACE TRIGGER mdhl_basic_trigger
  2    before INSERT OR UPDATE OR DELETE ON target_table for each row
  3  declare
  4
  5  my_bgroup VARCHAR2(3);
  6
  7    v_timer2 number := 0;
  8    v_timer number;
  9  BEGIN
 10
 11        IF INSERTING OR UPDATING THEN
 12          my_bgroup  := :NEW.BGROUP;
 13        ELSE
 14          my_bgroup  := :OLD.BGROUP;
 15        END IF;
 16
 17        INSERT INTO log_table VALUES(my_bgroup, 'BASIC_MDHL', current_timestamp, 'TinyString');
 18
 19  END mdhl_basic_trigger;
 20  /

SQL> insert into target_Table select 'ABC' from dual connect by level <= 10000;

10000 rows created.

Elapsed: 00:00:01.18

如果你知道你的字符串总是<32k,那么如果你创建你的触发器,你可以保持这个速度提升:

SQL> CREATE OR REPLACE TRIGGER mdhl_basic_trigger_compound
  2    FOR INSERT OR UPDATE OR DELETE ON target_table
  3
  4     COMPOUND TRIGGER
  5
  6     type events_rec is record (BGROUP VARCHAR2(3),
  7          TABLE_NAME VARCHAR2(255) ,
  8        EVENT_TS TIMESTAMP (7),
  9        EVENT_RAW varchar2(32767));
 10     TYPE EVENTS_HIST IS TABLE OF events_rec INDEX BY PLS_INTEGER;
 11     coll_events_hist EVENTS_HIST;
 12     ctr PLS_INTEGER := 0;
 13     my_bgroup VARCHAR2(3);
 14
 15  v_timer2 number := 0;
 16  v_timer number;
 17    BEFORE EACH ROW IS
 18      BEGIN
 19
 20        IF INSERTING OR UPDATING THEN
 21          my_bgroup  := :NEW.BGROUP;
 22        ELSE
 23          my_bgroup  := :OLD.BGROUP;
 24        END IF;
 25
 26        ctr := ctr + 1;
 27        coll_events_hist(ctr).BGROUP := my_bgroup;
 28        coll_events_hist(ctr).TABLE_NAME := 'BASIC_MDHL';
 29        coll_events_hist(ctr).EVENT_TS := current_timestamp;
 30        coll_events_hist(ctr).EVENT_RAW := 'TinyString';
 31
 32    END BEFORE EACH ROW;
 33
 34    AFTER STATEMENT IS
 35      BEGIN
 36  v_timer := dbms_utility.get_time;
 37        FORALL counter IN 1 .. coll_events_hist.count()
 38             INSERT INTO log_table VALUES coll_events_hist(counter);
 39  v_timer2 := v_timer2 + (dbms_utility.get_time - v_timer);
 40             dbms_output.put_line(v_timer2/100);
 41    END AFTER STATEMENT;
 42  END mdhl_basic_trigger_compound;
 43  /
SQL> insert into target_Table select 'ABC' from dual connect by level <= 10000;

10000 rows created.

Elapsed: 00:00:00.39

即。推迟操作直到插入。

答案 1 :(得分:1)

即使CLOB内联存储,与标准VARCHAR2相比也会产生一些开销,如appendix C of the LOB performance guideline中所述。

LOB的长度小于3964字节时,它将以36字节的标头内联存储。长度为VARCHAR2的X将被存储为X字节的数据,并带有额外的一个或两个字节的开销。

我认为这种开销将带入内存,这意味着PLSQL CLOB对象的效率将低于同等大小的VARCHAR2

34-35个额外字节将加起来,如以下脚本所示:

SQL> create table test_var(a varchar2(4000));

Table created

SQL> create table test_clob(a clob);

Table created

SQL> SET SERVEROUTPUT ON
SQL> DECLARE
  2    l_time TIMESTAMP := systimestamp;
  3  BEGIN
  4    FOR i IN 1..100000 LOOP
  5      INSERT INTO test_var VALUES (rpad('x', 1000, 'x'));
  6    END LOOP;
  7    dbms_output.put_line(systimestamp - l_time);
  8  END;
  9  /
+000000000 00:00:16.180299000

SQL> DECLARE
  2    l_time TIMESTAMP := systimestamp;
  3  BEGIN
  4    FOR i IN 1..100000 LOOP
  5      INSERT INTO test_clob VALUES (rpad('x', 1000, 'x'));
  6    END LOOP;
  7    dbms_output.put_line(systimestamp - l_time);
  8  END;
  9  /
+000000000 00:00:27.180716000

插入CLOB需要更多时间,这可以通过消耗的额外空间来解释:

SQL> EXEC dbms_stats.gather_table_stats(USER, 'TEST_VAR');

PL/SQL procedure successfully completed.

SQL> EXEC dbms_stats.gather_table_stats(USER, 'TEST_CLOB');

PL/SQL procedure successfully completed.

SQL> select blocks, table_name from user_tables where table_name like 'TEST_%';

    BLOCKS TABLE_NAME
---------- ------------------------------
     33335 TEST_CLOB
     28572 TEST_VAR

当我们插入较小的字符串时问题会更加严重:

-- after TRUNCATE tables
SQL> DECLARE
  2    l_time TIMESTAMP := systimestamp;
  3  BEGIN
  4    FOR i IN 1..1000000 LOOP
  5      INSERT INTO test_var VALUES (rpad('x', 10, 'x'));
  6    END LOOP;
  7    dbms_output.put_line(systimestamp - l_time);
  8  END;
  9  /

+000000000 00:00:51.916675000

SQL> DECLARE
  2    l_time TIMESTAMP := systimestamp;
  3  BEGIN
  4    FOR i IN 1..1000000 LOOP
  5      INSERT INTO test_clob VALUES (rpad('x', 10, 'x'));
  6    END LOOP;
  7    dbms_output.put_line(systimestamp - l_time);
  8  END;
  9  /

+000000000 00:01:57.377676000

-- Gather statistics

SQL> select blocks, table_name from user_tables where table_name like 'TEST_%';

    BLOCKS TABLE_NAME
---------- ------------------------------
      7198 TEST_CLOB
      2206 TEST_VAR