保持日期间隔不与Oracle + 11g中的触发器重叠并保持一致

时间:2014-08-12 11:20:55

标签: sql oracle plsql triggers oracle11g

假设我有以下内容:

    CREATE TABLE test (
    id NUMBER(10)
    , valid_from DATE
    , valid_to DATE,
    PRIMARY KEY (id, valid_from)
    );

    INSERT INTO test (id, valid_from) VALUES (1, '01/JAN/1900');
    INSERT INTO test (id, valid_from) VALUES (1, '01/JAN/1901');
    INSERT INTO test (id, valid_from) VALUES (1, '01/JAN/1902');
    INSERT INTO test (id, valid_from) VALUES (2, '01/JAN/1903');

输出:

            ID VALID_FROM VALID_TO 
    ---------- ---------- ---------
             1 01-JAN-01           
             1 01-JAN-02           
             2 01-JAN-03           
             1 01-JAN-00      

现在我需要一个触发器来保持VALID_TO字段与VALID_FROM一致,如下所示:

            ID VALID_FROM VALID_TO 
    ---------- ---------- ---------
             1 01-JAN-00  01-JAN-01
             1 01-JAN-01  01-JAN-02
             1 01-JAN-02           
             2 01-JAN-03 

我有一个计算VALID_TO的查询,并检查是否有任何需要更新的记录:

    WITH original AS (        
        SELECT id,
               valid_from,
               valid_to, 
               ROW_NUMBER() OVER (PARTITION BY id ORDER BY valid_from DESC) seq  
        FROM test
    ), should_be AS (
        SELECT df.id,
               df.valid_from AS VALID_FROM, 
               dt.valid_from AS VALID_TO
        FROM original df
        LEFT OUTER JOIN original dt ON (df.id = dt.id 
                                        AND df.seq = dt.seq + 1)
    ), update_req AS (
        SELECT
            should_be.*, 
            CASE WHEN original.VALID_TO = should_be.VALID_TO OR (original.VALID_TO IS NULL AND should_be.VALID_TO IS NULL) THEN 'N' ELSE 'Y' END UPDATE_REQUIRED        
        FROM should_be 
        INNER JOIN original ON (should_be.id = original.id AND should_be.valid_from = original.valid_from)
    )
    SELECT * 
    FROM update_req
    ORDER BY id, valid_from

输出:

            ID VALID_FROM VALID_TO  UPDATE_REQUIRED
    ---------- ---------- --------- ---------------
             1 01-JAN-00  01-JAN-01 Y              
             1 01-JAN-01  01-JAN-02 Y              
             1 01-JAN-02            N              
             2 01-JAN-03            N        

我在触发器中使用此查询,以确保VALID_TO字段在更新时更新:

    CREATE OR REPLACE TYPE ID_COLLECTION_T AS TABLE OF NUMBER(10);

    CREATE OR REPLACE TRIGGER trg_test
    FOR DELETE OR INSERT OR UPDATE
    ON test REFERENCING NEW AS NEW OLD AS OLD
    COMPOUND TRIGGER

        l_changed_ids ID_COLLECTION_T := ID_COLLECTION_T(); -- initialize

        AFTER EACH ROW IS
        BEGIN
            -- Keep track of changed ids
            CASE
                WHEN INSERTING OR UPDATING THEN l_changed_ids.extend; l_changed_ids(l_changed_ids.last) := :NEW.id;
                WHEN DELETING OR UPDATING THEN l_changed_ids.extend; l_changed_ids(l_changed_ids.last) := :OLD.id; 
            END CASE;

        END
        AFTER EACH ROW;

        AFTER STATEMENT IS

            l_existing_inconsistencies VARCHAR2(1);

        BEGIN

            -- first we check whether the executed statement caused any VALID_TO inconsistencies
            WITH original AS (        
                SELECT id,
                       valid_from,
                       valid_to, 
                       ROW_NUMBER() OVER (PARTITION BY id ORDER BY valid_from DESC) seq  
                FROM test
            ), should_be AS (
                SELECT df.id,
                       df.valid_from AS VALID_FROM, 
                       dt.valid_from AS VALID_TO
                FROM original df
                LEFT OUTER JOIN original dt ON (df.id = dt.id 
                                                AND df.seq = dt.seq + 1)
            ), update_req AS (
                SELECT
                    should_be.*, 
                    CASE WHEN original.VALID_TO = should_be.VALID_TO OR (original.VALID_TO IS NULL AND should_be.VALID_TO IS NULL) THEN 'N' ELSE 'Y' END UPDATE_REQUIRED        
                FROM should_be 
                INNER JOIN original ON (should_be.id = original.id AND should_be.valid_from = original.valid_from)
                WHERE original.id MEMBER OF l_changed_ids -- we ONLY (!) want to search for inconsistencies for modified ids
            )
            SELECT CASE WHEN 'Y' IN (SELECT UPDATE_REQUIRED FROM update_req) THEN 'Y' ELSE 'N' END
            INTO l_existing_inconsistencies
            FROM DUAL;

           -- If there are inconsistencies, then we update the table.
           IF l_existing_inconsistencies = 'Y' THEN 

                MERGE INTO test o
                USING (
                        WITH original AS (        
                            SELECT id,
                                   valid_from,
                                   valid_to, 
                                   ROW_NUMBER() OVER (PARTITION BY id ORDER BY valid_from DESC) seq  
                            FROM test
                        ), should_be AS (
                            SELECT df.id,
                                   df.valid_from AS VALID_FROM, 
                                   dt.valid_from AS VALID_TO
                            FROM original df
                            LEFT OUTER JOIN original dt ON (df.id = dt.id 
                                                            AND df.seq = dt.seq + 1)
                        )
                        SELECT
                                should_be.*, 
                                CASE WHEN original.VALID_TO = should_be.VALID_TO OR (original.VALID_TO IS NULL AND should_be.VALID_TO IS NULL) THEN 'N' ELSE 'Y' END UPDATE_REQUIRED        
                        FROM should_be 
                        INNER JOIN original ON (should_be.id = original.id AND should_be.valid_from = original.valid_from)
                        WHERE original.id MEMBER OF l_changed_ids -- we ONLY (!) want to search for inconsistencies for modified ids
                ) n
                ON (o.id = n.id AND o.valid_from = n.valid_from AND n.UPDATE_REQUIRED = 'Y')
                WHEN MATCHED THEN UPDATE SET o.valid_to = n.valid_to;

            END IF;

        END
        AFTER STATEMENT;

    END trg_test;

现在,触发器使插入/更新/删除的ID保持数据一致:

    INSERT INTO test (id, valid_from) VALUES (1, '01/JAN/1899');

现在我们在测试表中找到以下内容:

            ID VALID_FROM VALID_TO 
    ---------- ---------- ---------
             1 01-JAN-99  01-JAN-00
             1 01-JAN-00  01-JAN-01
             1 01-JAN-01  01-JAN-02
             1 01-JAN-02           
             2 01-JAN-03       

这里的问题是MEMBER OF声明。它会导致每个成员的全表扫描。 在多更新/插入语句的情况下,有许多潜在的ID已更改,因此l_changed_ids集合很大。 我无法优化成员: http://www.puthranv.com/search/label/Oracle%20Dynamic%20IN%20List http://www.oracle-developer.net/display.php?id=301

我试过了:

  1. 使用TABLE()转换集合对于许多单个插入/更新来说非常慢。
  2. 从循环中更新不一致的行是个坏主意,因为对于n-bulk语句,这可能会以递归的方式重新激活触发器n次。但是(!)使用循环非常快,因为每次都使用id上的索引。
  3. 我的问题是:

    1. 是否有其他触发器方法。
    2. 触发器必须适用于单个和批量更新/删除/插入语句。因此,如果存在不一致,则可能只递归执行一个额外的更新语句。
    3. 触发器根据id锁定行。 (这是后来的要求,但现在考虑它可能会很有趣。)
    4. UPDATE1:计算VALIT_TO日期的一些效果分析:

          -- Original query on 5mil records:  40 sec
          WITH original AS (        
              SELECT id,
                     valid_from,
                     valid_to, 
                     ROW_NUMBER() OVER (PARTITION BY id ORDER BY valid_from DESC) seq  
              FROM test
          ), should_be AS (
              SELECT df.id,
                     df.valid_from AS VALID_FROM, 
                     dt.valid_from AS VALID_TO
              FROM original df
              LEFT OUTER JOIN original dt ON (df.id = dt.id 
                                              AND df.seq = dt.seq + 1)
                                             ) select * from should_be
      
      
          -- TommCatt suggestion on 5mil records:  65 sec
          with Date_List as (
            select  t1.ID, t1.Valid_from as From_Date, Min( t2.Valid_from ) as To_Date 
            from    test t1
            left join test t2
              on    t2.id = t1.id
              and   t2.valid_from > t1.valid_from
            group by t1.ID, t1.Valid_from
          )
          select  id, from_date, to_date
          from    Date_List     
      
      
          -- TommCatt suggestion on 5mil records for 12c: untested
      
      
          -- a_horse_with_no_name suggestion on 5mil records:  10 sec WINNER!!
          SELECT id,
                 valid_from,
                 LEAD(valid_from, 1) OVER (PARTITION BY id ORDER BY valid_from ASC) valid_to
          FROM test
          -- EXEC Plan for the winner:
          ---------------------------------------------------------------------------------
          | Id  | Operation        | Name         | Rows  | Bytes | Cost (%CPU)| Time     |
          ---------------------------------------------------------------------------------
          |   0 | SELECT STATEMENT |              |  5106K|    63M| 22222   (1)| 00:04:27 |
          |   1 |  WINDOW BUFFER   |              |  5106K|    63M| 22222   (1)| 00:04:27 |
          |   2 |   INDEX FULL SCAN| SYS_C0011495 |  5106K|    63M| 22222   (1)| 00:04:27 |
          ---------------------------------------------------------------------------------                      
      

3 个答案:

答案 0 :(得分:2)

我碰巧在KScope14会议期间测试了SQL中的性能成员。我在博客上写了一些结果:

http://dspsd.blogspot.dk/2014/06/member-of-comparison-of-plsql-and-sql.html

尝试使用TABLE运算符替换MEMBER OF以将集合“转换”为“临时表”。这样的事情:

FROM should_be 
INNER JOIN original ON (should_be.id = original.id AND should_be.valid_from = original.valid_from)
INNER JOIN TABLE(l_changed_ids) chg ON (chg.column_value = original.id)

或者也许是这样:

FROM should_be 
INNER JOIN original ON (should_be.id = original.id AND should_be.valid_from = original.valid_from)
WHERE original.id IN (select column_value from TABLE(l_changed_ids))

如果您对添加基数提示的已更改ID的数量有近似想法,则可能对优化程序有用:

FROM should_be 
INNER JOIN original ON (should_be.id = original.id AND should_be.valid_from = original.valid_from)
WHERE original.id IN (select /*+ cardinality(42) */ column_value from TABLE(l_changed_ids))

以上是直接在这里输入的未经测试的代码 - 我希望你可以让它工作: - )


哦,对不起,我刚看了你的更新,你已经尝试了TABLE操作符,这对你来说很慢......

答案 1 :(得分:2)

基于触发器的另一种方法是在提交时使用物化视图刷新,在物化视图上使用约束。我看过一些讨论和论坛中提到的方法。这里给出了一个例子和一些讨论:

http://jeffkemponoracle.com/2012/08/30/non-overlapping-dates-constraint/

我自己没试过,但可能值得研究一下?

答案 2 :(得分:2)

您可以通过与任何其他计划相媲美的表现轻松地模拟您的出路。当然,维护工作将大大减少。我自己使用这种技术并取得了很好的效果。

首先,当你有这样的From / To字段集时,你可以设置我所谓的行跨越依赖关系。从数据完整性的角度来看,这很糟糕。每次执行DML时,都必须至少执行两个语句。要插入新的有效日期记录,您必须找到" current"记录并更新" to_date"然后发出插入。任何日期的任何更新只能通过两个更新语句来完成。而且,正如你可以清楚地看到的那样,保持日期序列的有效性绝对是一场噩梦。

解决方案真的很简单。只有"有效"或"有效"日期而不是From_Date字段。完全删除To_date字段。现在,让我们规定当ID在有效字段中的日期变为有效时,它将保持有效,直到输入具有相同ID和更晚日期的另一行。第二行中的日期字段是行变为有效的日期,但它也是第一行变为无效(或不再有效 - 我更喜欢的术语)的点。

一个插页。完成!

因此,重叠和间隙变得不可能。你甚至不必检查它们。不可能!

但是有些人会想要看到"来自"和" To"在他们的报告中,对吗?没关系。 "从"和" To"结果集很好,他们只是臭作为数据。所以这里是如何获得"来自"和" To"来自数据:

with
Date_List( id, from_date, to_date )as(
  select  t1.ID, t1.Valid as From_Date, Min( t2.Valid ) as To_Date 
  from    test t1
  left join test t2
    on    t2.id = t1.id
    and   t2.valid > t1.valid
  group by t1.ID, t1.Valid
)
select  id, from_date, to_date
from    Date_List
order by id, From_date desc;

在cte你加入PK到PK - 非常快。在cte之外,您可能需要再次加入表格以获取其他数据,我确定您为了清楚而省略了这些数据。这仍然很快,因为你再次加入PK领域。获得Oracle-12c后,您可以像这样重写它:

select  t1.id, t1.valid as from_date, t2.valid as To_date -- t1.etc, ...
from    test t1
left join test t2
  on    t2.id = t1.id
  and   t2.valid =(
            select  Min( t3.valid )
            from    test t3
            where   t3.id = t1.id
                and t3.valid > t1.valid )
order by t1.id, t1.valid desc;

在某种程度上,使用连接和子查询会更糟糕。然而,时序测试将显示出令人印象深刻的结果。但是,即使在表格中添加一个To_Date字段以获得更好的性能,请记住我之前说的:GAPS和重叠是不可能的!如果你尝试的话,你甚至无法搞砸。你可以想象的最糟糕的情况是对同一个ID输入相同的日期两次,但是由于这些定义了PK,系统不会让你这样做。想想你不必写的所有触发器,约束和存储过程(不要保持日期同步)!