在登台表中使用SQL * Loader然后在主数据库表中发出加载数据

时间：2015-09-15 08:52:08

标签： sql oracle plsql oracle11g

我正在尝试使用SQL * Loader将数据加载到主数据库表中。数据将提供给管道分离的CSV文件。

我开发了一个shell脚本来加载数据，除了一件事情之外它工作得很好。

以下是重新创建问题的详细信息和数据。暂存表结构，其中将使用SQL / * Loader填充数据：

stg_cmts_data

CSV文件中的数据;对于cmts_data.csv中的wnlb-cmts-01-1|10.15.0.1 wnlb-cmts-02-2|10.15.16.1 wnlb-cmts-03-3|10.15.48.1 wnlb-cmts-04-4|10.15.80.1 wnlb-cmts-05-5|10.15.96.1 ：

stg_dhcp_data

dhcp_map_03092015_1.csv中的dhcp-1-1-1|10.25.23.10,25.26.14.01 dhcp-1-1-2|56.25.111.25,100.25.2.01 dhcp-1-1-3|25.255.3.01,89.20.147.258 dhcp-1-1-4|10.25.26.36,200.32.58.69 dhcp-1-1-5|80.25.47.369,60.258.14.10 ：

stg_link_data

link_data.csv中的dhcp-1-1-1|wnlb-cmts-01-1,wnlb-cmts-02-2 dhcp-1-1-2|wnlb-cmts-03-3,wnlb-cmts-04-4,wnlb-cmts-05-5 dhcp-1-1-3|wnlb-cmts-01-1 dhcp-1-1-4|wnlb-cmts-05-8,wnlb-cmts-05-6,wnlb-cmts-05-0,wnlb-cmts-03-3 dhcp-1-1-5|wnlb-cmts-02-2,wnlb-cmts-04-4,wnlb-cmts-05-7 wnlb-dhcp-1-13|wnlb-cmts-02-2 ：

create table subntwk (subntwk_nm varchar2(20), subntwk_ip varchar2(30));
create table link (link_nm varchar2(50));

现在将这些数据加载到临时表后，我必须填充主数据库表：

spool load_cmts.log
set serveroutput on

DECLARE
  CURSOR c_stg_cmts IS SELECT *
                     FROM stg_cmts_data;
  TYPE t_stg_cmts IS TABLE OF stg_cmts_data%ROWTYPE INDEX BY pls_integer;
  l_stg_cmts t_stg_cmts;
  l_cmts_cnt       NUMBER;
  l_cnt            NUMBER;
  l_cnt_1          NUMBER;
 BEGIN
   OPEN c_stg_cmts;
   FETCH c_stg_cmts BULK COLLECT INTO l_stg_cmts;
   FOR i IN l_stg_cmts.FIRST..l_stg_cmts.LAST
   LOOP
      SELECT COUNT (1)
        INTO l_cmts_cnt
        FROM subntwk
       WHERE subntwk_nm = l_stg_cmts(i).cmts_token;

     IF l_cmts_cnt < 1 THEN
       INSERT
       INTO SUBNTWK
         (
           subntwk_nm
         )
         VALUES
        (
          l_stg_cmts(i).cmts_token      
         );

       DBMS_OUTPUT.PUT_LINE('Token has been added : '||l_stg_cmts(i).cmts_token);
     ELSE    
       DBMS_OUTPUT.PUT_LINE('Token is already present');
     END IF;

     EXIT WHEN l_stg_cmts.COUNT =0;
   END LOOP;

   commit;
 EXCEPTION
   WHEN OTHERS THEN
     DBMS_OUTPUT.put_line ('ERROR OCCURED ' || SQLERRM);
 END;
 /

 exit

我为加载数据而创建的SQL脚本：

dhcp

spool load_dhcp.log set serveroutput on DECLARE CURSOR c_stg_dhcp IS SELECT * FROM stg_dhcp_data; TYPE t_stg_dhcp IS TABLE OF stg_dhcp_data%ROWTYPE INDEX BY pls_integer; l_stg_dhcp t_stg_dhcp; l_dhcp_cnt NUMBER; l_cnt NUMBER; l_cnt_1 NUMBER; BEGIN OPEN c_stg_dhcp; FETCH c_stg_dhcp BULK COLLECT INTO l_stg_dhcp; FOR i IN l_stg_dhcp.FIRST..l_stg_dhcp.LAST LOOP SELECT COUNT (1) INTO l_dhcp_cnt FROM subntwk WHERE subntwk_nm = l_stg_dhcp(i).dhcp_token; IF l_dhcp_cnt < 1 THEN INSERT INTO SUBNTWK ( subntwk_nm ) VALUES ( l_stg_dhcp(i).dhcp_token ); DBMS_OUTPUT.PUT_LINE('Token has been added : '||l_stg_dhcp(i).dhcp_token); ELSE DBMS_OUTPUT.PUT_LINE('Token is already present'); END IF; EXIT WHEN l_stg_dhcp.COUNT =0; END LOOP; commit; EXCEPTION WHEN OTHERS THEN DBMS_OUTPUT.put_line ('ERROR OCCURED ' || SQLERRM); END; / exit ：

spool load_link.log
set serveroutput on

DECLARE
   l_cmts_1           VARCHAR2( 4000 CHAR );
   l_cmts_add         VARCHAR2( 200 CHAR );
   l_dhcp_cnt         NUMBER;
   l_cmts_cnt         NUMBER;
   l_link_cnt          NUMBER;
   l_add_link_nm      VARCHAR2( 200 CHAR );
 BEGIN
   FOR r IN (
        SELECT dhcp_token, cmts_to_add || ',' cmts_add
          FROM stg_link_data
       )
    LOOP

l_cmts_1      := r.cmts_add;
l_cmts_add    := TRIM(SUBSTR( l_cmts_1, 1, INSTR(l_cmts_1, ',' ) - 1 ));

SELECT COUNT(1)
  INTO l_dhcp_cnt
  FROM subntwk
 WHERE subntwk_nm = r.dhcp_token;

 IF l_dhcp_cnt = 0 THEN
  DBMS_OUTPUT.PUT_LINE('Device not found : '|| r.dhcp_token);
 ELSE
  WHILE l_cmts_add IS NOT NULL
  LOOP
    l_add_link_nm := r.dhcp_token||'_TO_'||l_cmts_add;
    SELECT COUNT(1)
      INTO l_cmts_cnt
      FROM subntwk
     WHERE subntwk_nm = TRIM(l_cmts_add);

    SELECT COUNT(1)
      INTO l_link_cnt
      FROM link
     WHERE link_nm = l_add_link_nm;

    IF l_cmts_cnt > 0 AND l_link_cnt = 0 THEN
      INSERT INTO link ( link_nm)
                VALUES ( l_add_link_nm);
      DBMS_OUTPUT.PUT_LINE( l_add_link_nm ||' '||'Has been added.');
    ELSIF l_link_cnt > 0 THEN
      DBMS_OUTPUT.PUT_LINE( 'link is already present : '|| l_add_link_nm );
   ELSIF l_cmts_cnt = 0 then
     DBMS_OUTPUT.PUT_LINE( 'NO CMTS FOUND for device to create link : '|| l_cmts_add );
    END IF;
  l_cmts_1   := TRIM(SUBSTR( l_cmts_1, INSTR( l_cmts_1, ',' ) + 1 ));
  l_cmts_add := TRIM(SUBSTR( l_cmts_1, 1, INSTR( l_cmts_1, ',' ) - 1 ));
END LOOP;
END IF;
  END LOOP;
  COMMIT;
EXCEPTION
   WHEN OTHERS THEN
    DBMS_OUTPUT.put_line ('ERROR OCCURED ' || SQLERRM);  
 END;
/
exit

链接：

LOAD DATA
INFILE 'cmts_data.csv'
APPEND
INTO TABLE STG_CMTS_DATA
when (cmts_token != '') AND (cmts_token != 'NULL') AND (cmts_token != 'null')
  and (cmts_ip != '') AND (cmts_ip != 'NULL') AND (cmts_ip != 'null')
  FIELDS TERMINATED BY '|' OPTIONALLY ENCLOSED BY '"'
 TRAILING NULLCOLS
(cmts_token                "RTRIM(LTRIM(:cmts_token))",
 cmts_ip                   "RTRIM(LTRIM(:cmts_ip))")

控制文件：

dhcp

LOAD DATA INFILE 'dhcp_data.csv' APPEND INTO TABLE STG_DHCP_DATA when (dhcp_token != '') AND (dhcp_token != 'NULL') AND (dhcp_token != 'null') and (dhcp_ip != '') AND (dhcp_ip != 'NULL') AND (dhcp_ip != 'null') FIELDS TERMINATED BY '|' OPTIONALLY ENCLOSED BY '"' TRAILING NULLCOLS (dhcp_token "RTRIM(LTRIM(:dhcp_token))", dhcp_ip "RTRIM(LTRIM(:dhcp_ip))") ：

 LOAD DATA
 INFILE 'link_data.csv'
 APPEND
 INTO TABLE STG_LINK_DATA
 when (dhcp_token != '') AND (dhcp_token != 'NULL') AND (dhcp_token != 'null')
   and (cmts_to_add != '') AND (cmts_to_add != 'NULL') AND (cmts_to_add != 'null')
   FIELDS TERMINATED BY '|' OPTIONALLY ENCLOSED BY '"'
 TRAILING NULLCOLS
 (dhcp_token                   "RTRIM(LTRIM(:dhcp_token))",
  cmts_to_add          CHAR(4000) "RTRIM(LTRIM(:cmts_to_add))")

链接：

 if [ ! -d ./log ]
 then
   mkdir log
 fi

 if [ ! -d ./done ]
 then
    mkdir done
 fi

 if [ ! -d ./bad ]
 then
   mkdir bad
 fi

 nohup time sqlldr username/password@SID CONTROL=load_cmts_data.ctl    LOG=log/ldr_cmts_data.log BAD=log/ldr_cmts_data.bad DISCARD=log/ldr_cmts_data.reject ERRORS=100000 DIRECT=TRUE PARALLEL=TRUE &
 nohup time username/password@SID @load_cmts.sql

 nohup time sqlldr username/password@SID CONTROL=load_dhcp_data.ctl LOG=log/ldr_dhcp_data.log BAD=log/ldr_dhcp_data.bad DISCARD=log/ldr_dhcp_data.reject ERRORS=100000 DIRECT=TRUE PARALLEL=TRUE &                       
 nohup time sqlplus username/password@SID @load_dhcp.sql

 nohup time sqlldr username/password@SID CONTROL=load_link_data.ctl LOG=log/ldr_link_data.log BAD=log/ldr_link_data.bad DISCARD=log/ldr_link_data.reject ERRORS=100000 DIRECT=TRUE PARALLEL=TRUE &   
 nohup time sqlplus username/password@SID @load_link.sql

 mv *.log ./log

Shell脚本：

link

所以这里我遇到的问题是在将数据加载到subntwk表时我正在检查link表中是否存在DHCP，然后继续进行其他日志错误。如果CMTS退出，则创建链接其他日志错误。

现在我们可以在这里将多个CMTS与单个DHCP相关联。

所以这里在stg_link_data表中创建链接，但是对于循环的最后一次迭代，我从dhcp-1-1-1|wnlb-cmts-01-1,wnlb-cmts-02-2 表中获取单独的逗号分隔的CMTS，它给出了我没有找到CMTS的日志。 / p>

例如

dhcp-1-1-1

在这里，我应该使用wnlb-cmts-01-1和wnlb-cmts-02-2

创建subntwk的链接

这些数据都存在于wnlb-cmts-02-2 NOT FOUND表格中，但它仍以subntwk为对象，但我们已将其加载到stg_link_data表中。

同样的情况发生在 link is already present : dhcp-1-1-1_TO_wnlb-cmts-01-1 NO CMTS FOUND for device to create link : wnlb-cmts-02-2 link is already present : dhcp-1-1-2_TO_wnlb-cmts-03-3 link is already present : dhcp-1-1-2_TO_wnlb-cmts-04-4 NO CMTS FOUND for device to create link : wnlb-cmts-05-5 NO CMTS FOUND for device to create link : wnlb-cmts-01-1 NO CMTS FOUND for device to create link : wnlb-cmts-05-8 NO CMTS FOUND for device to create link : wnlb-cmts-05-6 NO CMTS FOUND for device to create link : wnlb-cmts-05-0 NO CMTS FOUND for device to create link : wnlb-cmts-03-3 link is already present : dhcp-1-1-5_TO_wnlb-cmts-02-2 link is already present : dhcp-1-1-5_TO_wnlb-cmts-04-4 NO CMTS FOUND for device to create link : wnlb-cmts-05-7 Device not found : wnlb-dhcp-1-13 表中的所有CMTS中（最后一个）（我相信你得到了我想解释的内容）。

但是，当我单独在SQL Developer中运行SQL脚本时，它会将所有有效链接插入到链接表中。

这里应该在链接表中创建9行，而现在只创建5行。

我也在我的脚本中使用COMMIT，但它没有帮助我。

请在您的计算机上运行这些脚本，并告诉我您是否也获得了与我相同的行为，请为我提供一些解决方案。我从昨天开始尝试了很多东西，但它仍然是一样的。

以下是链接表的日志：

{{1}}

如果需要更多信息，请告知我们。

1 个答案:

答案 0 :(得分：0)

我晚些时候发现，在使用UNIX机器加载到临时表时，它为每一行创建了新行。这就是找不到最后一个CMTS的原因，为此我使用DOS 2 UNIX转换，它开始完美运行。

那是dos2unix错误！

相关问题
最新问题