Question

我正在使用批量收集来缩短执行时间。当我不使用批量收集时，它将在4分钟内执行。但是，当我使用批量收集时，没有任何输出，控制台中都不会显示错误消息。我可以看到创建了一个空白后台打印文件。请让我知道如果我没有正确利用批量收集，我们也可以在带限制的select语句中使用此子句吗？该表最多包含100万条记录。

SET SERVEROUTPUT ON FORMAT WRAPPED
SET VERIFY OFF
SET FEEDBACK OFF
SET TERMOUT OFF

SPOOL C:\Temp\spool_1.txt

DECLARE

  cursor c2 is (
    select count(distinct e.cdb_pref_event_id)
          ,e.supp_cd
      from (select distinct eh.cdb_customer_id   cdb_customer_id
                           ,eh.cdb_pref_event_id cdb_pref_event_id
                           ,eh.supp_cd           supp_cd
              from (select *
                      from cdb_stg.cpm_pref_event_stg_arc
                     where trunc(load_date) = trunc(sysdate - 1)) eh
              Left outer join cdb_admin.cpm_pref_result er on (eh.cdb_customer_id =
                                                              er.cdb_customer_id and
                                                              eh.cdb_pref_event_id =
                                                              er.cdb_pref_event_id)
             where er.cdb_pref_event_id is null
               and er.cdb_customer_id is null) r
      join cdb_admin.cpm_pref_event_exception e on (r.cdb_customer_id =
                                                   e.cdb_customer_id and
                                                   r.cdb_pref_event_id =
                                                   e.cdb_pref_event_id)
     group by e.supp_cd);

  TYPE totalprefresults is table of NUMBER(20);
  TYPE supcd_1 is table of cdb_admin.cpm_pref_event_stg.supp_cd%TYPE;
  total_prefresults totalprefresults;
  supcd1            supcd_1;
  --Total_prefresults NUMBER(20);
  --SUPCD1 CDB_ADMIN.CPM_PREF_EVENT_STG.supp_cd%TYPE;
  profile_counts NUMBER(20);

  iter Integer := 0;

BEGIN

  select count(distinct cdb_customer_id)
    into profile_counts
    from cdb_admin.cpm_pref_event_exception h
   where cdb_customer_id in
         (Select distinct e.cdb_customer_id
            from (Select distinct eh.cdb_customer_id   cdb_customer_id
                                 ,eh.cdb_pref_event_id cdb_pref_event_id
                                 ,eh.supp_cd           supp_cd
                    from (select *
                            from cdb_stg.cpm_pref_event_stg_arc
                           where trunc(load_date) = trunc(sysdate - 1)) eh
                    Left outer join cdb_admin.cpm_pref_result er on (eh.cdb_customer_id =
                                                                    er.cdb_customer_id and
                                                                    eh.cdb_pref_event_id =
                                                                    er.cdb_pref_event_id)
                   where er.cdb_pref_event_id is null
                     and er.cdb_customer_id is null) r
            join cdb_admin.cpm_pref_event_exception e on (r.cdb_customer_id =
                                                         e.cdb_customer_id and
                                                         r.cdb_pref_event_id =
                                                         e.cdb_pref_event_id)
           where e.supp_cd = 'PROFILE-NOT-FOUND')
     and h.supp_cd != 'PROFILE-NOT-FOUND';

  dbms_output.put_line('TOTAL EVENTS VALIDATION');
  dbms_output.put_line('-------------------------------------------------------------');
  dbms_output.put_line('');

  dbms_output.put_line(rpad('Pref_Counts', 25) || rpad('Supp_CD', 25));

  OPEN c2;
  LOOP
    FETCH c2 BULK COLLECT
      INTO total_prefresults
          ,supcd1 limit 100;
    EXIT WHEN c2%NOTFOUND;
    dbms_output.put_line(rpad(total_prefresults, 25) || rpad(supcd1, 25));

    IF (supcd1 = 'PROFILE-NOT-FOUND')
    then
      dbms_output.put_line('');
      dbms_output.put_line('Profile not found records count : ' ||
                           total_prefresults);

      dbms_output.put_line(profile_counts ||
                           ' : counts moved to other exceptions ');
      dbms_output.put_line((total_prefresults - profile_counts) ||
                           ' : are still in Profile_not_found exception');

    END IF;

    iter := iter + 1;
  END LOOP;
  CLOSE c2;
  dbms_output.put_line('');
  dbms_output.put_line('Number of missing Records: ' || iter);

END;
/
SPOOL OFF

Answer 1

我认为瓶颈是这种情况：where trunc(load_date) = trunc(sysdate - 1)

您没有在trunc(load_date)上建立索引吗？在trunc(load_date)上创建基于函数的索引，或者在load_date上创建索引，然后尝试

WHERE load_date >= trunc(sysdate - 1) AND load_date < trunc(sysdate)

还要检查您的查询是否确实需要distinct。如有可能，将其删除。

Answer 2

我已将您的代码从OPEN c2;重组为CLOSE c2;

BULK COLLECT应该只在一次（一次执行）中将所有数据存储在集合中，然后可以使用FOR loop中的索引（即以下情况中的I）使用该集合如下：

OPEN C2; 
FETCH C2 BULK COLLECT INTO
    TOTAL_PREFRESULTS,
    SUPCD1;
--EXIT WHEN C2%NOTFOUND;
CLOSE C2;

-- To list down all the values before processing the logic
FOR I IN TOTAL_PREFRESULTS.FIRST..TOTAL_PREFRESULTS.LAST LOOP
DBMS_OUTPUT.PUT_LINE(RPAD(TOTAL_PREFRESULTS(I), 25)
                     || RPAD(SUPCD1(I), 25));
END LOOP;


FOR I IN TOTAL_PREFRESULTS.FIRST..TOTAL_PREFRESULTS.LAST LOOP
    IF ( SUPCD1(I) = 'PROFILE-NOT-FOUND' ) THEN
        DBMS_OUTPUT.PUT_LINE('');
        DBMS_OUTPUT.PUT_LINE('Profile not found records count : ' || TOTAL_PREFRESULTS(I));
        DBMS_OUTPUT.PUT_LINE(PROFILE_COUNTS || ' : counts moved to other exceptions ');
        DBMS_OUTPUT.PUT_LINE((TOTAL_PREFRESULTS(I) - PROFILE_COUNTS)
                             || ' : are still in Profile_not_found exception');
    END IF;

    ITER := ITER + 1;
END LOOP;

在代码中替换上面的代码段，然后尝试执行。

请参阅guide to use BULK COLLECT

干杯！

Answer 3

批量收集可以显着提高性能。但是，有一些陷阱。
首先，％notfound的含义有所不同。在标准游标上，％notfound表示所有行均已获取，没有更多行了。对于批量收集，此更改为“没有足够的行达到指定的LIMIT（如果存在）。这并不意味着就没有达到仅达到指定限制的行。例如，如果您的限制为100，而提取仅检索了50，则％notfound将返回True。这是参考指南失败的地方。
第二个是没有limit子句的情况：来自游标的所有行都返回到共享内存（PGA？）。那么这是什么问题。如果有100行或1000行，那么很可能是k，但是假设有100,000行或1M行，它们仍然都已加载到内存中。最后（至少现在），当使用limit子句时，整个Fetch + Process必须本身被包含在循环中，或者您仅处理第一个fetch（仅指指定的限制行数），无论实际上有多少行。参考指南失败的另一点。以下骨骼容纳了以上内容。

    declare 
       max_bulk_rows constant integer  := 1000;  -- define the max number of rows for each fetch ... 

       cursor c_bulk is(
        Select ... ;

       type bulk_row_t is table of c_bulk%rowtype; 
       bulk_row  bulk_row_t; 

    Begin 
       open c_bulk;  
       loop
           fetch c_bulk                      -- fill buffer 
           bulk collect into  bulk_row
           limit max_bulk_row; 

           for i in bulk_row.first .. bulk_row.last -- process each row in buffer 
           loop
               "process individual row here"
           end loop; 

           foreach ...                      -- bulk output of rows here is needed.

           exit when bulk_row.count < max_bulk_row;  -- exit process loop if all rows processed

       end loop ;   -- loop back and fetch next buffer if needed
   close c_bulk;
    ...
   end;

使用批量收集来提高plsql的性能

3 个答案: