Oracle数据库测试挑战 - 比较模式数据

时间:2014-03-28 01:47:34

标签: oracle

想知道是否有人可以帮助我指出正确的方向,或者告诉我,我通过sql尝试这个很疯狂。如果sql太具挑战性,是否有任何免费或廉价的工具可以帮助我实现自动化?

我正在测试旧的和新的Oracle数据库之间的一些数据。我想做的是能够为模式中的所有表动态生成此查询。

Select Column_1, Column_2 FROM Table_1
MINUS
Select Column_1, Column_2 FROM Table_1@"OLD_SERVER"

一个问题是,为每个表选择的列应该只是不以' ETL'开头的列。因为这些预计会随着迁移而改变。

为了保持动态,我可以使用all_tab_columns循环遍历每个表吗?

因此,对于一个简化示例,我们假设此查询返回了以下结果,并且您可以预期ALL_TAB_COLUMNS的结果在OLD和NEW数据库之间是相同的:

select TABLE_NAME, COLUMN_NAME from ALL_TAB_COLUMNS where owner = 'OWNER1'

TABLE_NAME, COLUMN_NAME
-----------------------
TABLE1, COLUMN_1
TABLE1, COLUMN_2
TABLE1, ETLCOLUMN_3

TABLE2, COLUMN_A
TABLE2, COLUMN_B
TABLE2, ETLCOLUMN_C

如何在旧数据库和新数据库上编写将在相同表和列(不以ETL开头)之间运行减号的查询,并输出结果以及表名和运行日期,以及然后循环到下一个表并做同样的事情?

2 个答案:

答案 0 :(得分:2)

首先 - 看看这个: http://docs.oracle.com/cd/E11882_01/server.112/e41481/spa_upgrade.htm#RATUG210

第二 - 您想编写一个发出查询的查询 - 问题是在user_tab_columns中每列都是一行。 为此,我建议你阅读:http://www.dba-oracle.com/t_converting_rows_columns.htm
您的源表是USER_TAB_COLUMNS,在运行查询时,您可以添加一个说"其中column_name不喜欢' ETL%'等等 之后 - 查询看起来像:
选择'选择'

 || listagg..... (from the link) || 'from table name' sql 
from user_tab_columns
where column_name not like 'ETL%'
and table_name = 'table name'
group by table_name

并且顺便说一句 - 你并不疯狂 - 在更换系统之前,你需要能够签署升级才能成功 - 这是唯一的方法。

顺便说一句 - 如果您更深入地描述系统和升级 - 我相信社区将能够帮助您找到更深入地测试它的方法,并会指出您要测试的东西。
在许多情况下仅测试输出是不够的....

好运!

答案 1 :(得分:0)

此测试可以使用SQL和PL / SQL自动执行。这样做你并不疯狂。像这样的比较系统对于测试复杂系统的变化非常有用。它不如自动化单元测试好,但它可以显着增强典型的数据库测试。

以下代码是一个完整的示例。但在现实世界中,有许多问题可能需要几天才能解决。例如,处理CLOB,大表,时间戳和基于序列的值等

示例模式和数据差异

create user schema1 identified by schema1;
create user schema2 identified by schema2;
alter user schema1 quota unlimited on users;
alter user schema2 quota unlimited on users;

--Data in 1, not 2.
create table schema1.table1 as select 1 a, 1 b from dual;
create table schema2.table1(a number, b number);

--Data in 2, not 1.
create table schema1.table2(a number, b number);
create table schema2.table2 as select 1 a, 1 b from dual;

--Same data in both, excluding unused column.
create table schema1.table3 as select 1 a, 1 b, 'asdf' ETL_c from dual;
create table schema2.table3 as select 1 a, 1 b, 'fdsa' ETL_c from dual;

--Table DDL difference.
create table schema1.table4(a number);
create table schema2.table4(b number);

--Privileges can be tricky.
grant select on dba_tab_columns to <your schema>;

打印差异脚本的步骤

create or replace procedure print_differences(
    p_old_schema in varchar2,
    p_new_schema in varchar2) authid current_user
is
    v_table_index number := 0;
    v_row_count number;
begin
    --Print header information.
    dbms_output.put_line('--Comparison between '||p_old_schema||' and '||
        p_new_schema||', at '||to_char(sysdate, 'YYYY-MM-DD HH24:MI')||'.'||chr(10));

    --Create a SQL statement to return the differences for each table.
    for differences in
    (
        --Return number of differences and SQL statements to view them.
        select
        '
        with old_table as (select '||column_list||' from '||p_old_schema||'.'||table_name||')
        ,    new_table as (select '||column_list||' from '||p_new_schema||'.'||table_name||')
        select * from
        (
            select ''OLD'' old_or_new, old_table.* from old_table minus
            select ''OLD'' old_or_new, new_table.* from new_table
        )
        union all
        select * from
        (
            select ''NEW'' old_or_new, new_table.* from new_table minus
            select ''NEW'' old_or_new, old_table.* from old_table
        )
        ' difference_sql, table_name
        from
        (
            select table_name
                ,listagg(column_name, ',') within group (order by column_id) column_list
            from dba_tab_columns
            where owner = p_old_schema
                and column_name not like 'ETL%'
            group by table_name
        ) column_lists
    ) loop
        begin
            --Print table information:
            v_table_index := v_table_index+1;
            dbms_output.put_line(chr(10)||'--'||lpad(v_table_index, 3, '0')||':  '||differences.table_name);

            --Count differences.
            execute immediate 'select count(*) from ('||differences.difference_sql||')' into v_row_count;

            --Print SQL statements to investigate differences.
            if v_row_count = 0 then
                dbms_output.put_line('--No differences.');
            else
                dbms_output.put_line('--Differences: '||v_row_count);
                dbms_output.put_line(differences.difference_sql||';');
            end if;
        exception when others then
            dbms_output.put_line('/*Error with this statement, possible DDL difference: '
                ||differences.difference_sql||dbms_utility.format_error_stack||
                dbms_utility.format_error_backtrace||'*/');
        end;
    end loop;
end;
/

运行程序

begin
    print_differences('SCHEMA1', 'SCHEMA2');
end;
/

示例输出

该程序不输出实际差异。如果存在差异,则会输出一个显示差异的脚本。使用体面的IDE,这将是一种更好的查看数据的方式,它还有助于进一步分析差异。

--Comparison between SCHEMA1 and SCHEMA2, at 2014-03-28 23:44.


--001:  TABLE1
--Differences: 1

        with old_table as (select A,B from SCHEMA1.TABLE1)
        ,    new_table as (select A,B from SCHEMA2.TABLE1)
        select * from
        (
            select 'OLD' old_or_new, old_table.* from old_table minus
            select 'OLD' old_or_new, new_table.* from new_table
        )
        union all
        select * from
        (
            select 'NEW' old_or_new, new_table.* from new_table minus
            select 'NEW' old_or_new, old_table.* from old_table
        )
        ;

--002:  TABLE2
--Differences: 1

        with old_table as (select A,B from SCHEMA1.TABLE2)
        ,    new_table as (select A,B from SCHEMA2.TABLE2)
        select * from
        (
            select 'OLD' old_or_new, old_table.* from old_table minus
            select 'OLD' old_or_new, new_table.* from new_table
        )
        union all
        select * from
        (
            select 'NEW' old_or_new, new_table.* from new_table minus
            select 'NEW' old_or_new, old_table.* from old_table
        )
        ;

--003:  TABLE3
--No differences.

--004:  TABLE4
/*Error with this statement, possible DDL difference: 
        with old_table as (select A from SCHEMA1.TABLE4)
        ,    new_table as (select A from SCHEMA2.TABLE4)
        select * from
        (
            select 'OLD' old_or_new, old_table.* from old_table minus
            select 'OLD' old_or_new, new_table.* from new_table
        )
        union all
        select * from
        (
            select 'NEW' old_or_new, new_table.* from new_table minus
            select 'NEW' old_or_new, old_table.* from old_table
        )
        ORA-06575: Package or function A is in an invalid state
ORA-06512: at "JHELLER.PRINT_DIFFERENCES", line 48
*/