skip / ignore /不插入重复的行

时间:2012-08-30 18:55:45

标签: sql sql-server-2008 insert merge

这就是我想要的: 将行从tableA复制到tableB,但不复制tableB中已有的任何行。没有PK或识别,我只想检查每个字段,如果匹配的行包含所有相同的字段,则不插入。我试过INSERT和MERGE:

 --try with INSERT
 INSERT TableB(col1,col2,col3,col4,col5,col6, etc.)
 SELECT (col1,col2,col3,col4,col5,col6, etc.)  
 FROM tableA as src
 WHERE NOT EXISTS (SELECT 1 FROM TableB as T
            WHERE               
            T.col1 = src.col1 AND
            T.col2 = src.col2 AND
            T.col3 = src.col3 AND
            T.col4 = src.col4 AND
            T.col5 = src.col5 AND
            T.col6 = src.col6, etc.)

我也尝试过MERGE:

  MERGE INTO tableA src
  USING tableB T
  ON (
            T.col1 = src.col1 AND
            T.col2 = src.col2 AND
            T.col3 = src.col3 AND
            T.col4 = src.col4 AND
            T.col5 = src.col5 AND
            T.col6 = src.col6, etc.)
  WHEN NOT MATCHED THEN
  INSERT (col1,col2,col3,col4,col5,col6, etc.)
  VALUES (col1,col2,col3,col4,col5,col6, etc.);

两者都完全相同:我会进行插入(或合并),最初所有行都被导入,这很好,但是第二次尝试(应该插入/合并0行)它会插入/合并60%的行。

我认为这个问题是因为我没有PK,这是我在其他帖子上看到的。我只是想在所有领域匹配,这可能吗?我错过了什么吗?

感谢您的任何建议/指示!

4 个答案:

答案 0 :(得分:6)

您可以使用EXCEPT

INSERT INTO table1
SELECT * from table2
EXCEPT 
SELECT * from table1;

看看小提琴here

答案 1 :(得分:2)

您可以在select语句中使用LEFT OUTER JOIN来查找所有列都为NULL的记录,如下所示:

 INSERT INTO tableB (col1,col2,col3,col4,col5,col6, etc.)
 SELECT (A.col1, A.col2, A.col3, A.col4, A.col5, A.col6, etc.)  
 FROM tableA as A
 LEFT OUTER JOIN tableB AS B ON A.col1 = B.col1 AND A.col2 = B.col2 AND 
     A.col3 = B.col3 ... /* all the way to B.col99 or whatever */
 WHERE B.col1 IS NULL AND B.col2 IS NULL AND B.col3 IS NULL ... 
    /* all the way to B.col99 or whatever your last column is */

答案 2 :(得分:2)

我的猜测是问题是NULL。尝试查询:

INSERT TableB(col1,col2,col3,col4,col5,col6, etc.)
SELECT (col1,col2,col3,col4,col5,col6, etc.)  
FROM tableA as src
WHERE NOT EXISTS (SELECT 1 FROM TableB as T
                  WHERE (T.col1 = src.col1 or t.col1 is null and src.col1 is null) AND
                        (T.col2 = src.col2 or t.col2 is null and src.col2 is null) AND
                        . . .

答案 3 :(得分:0)

您的问题可能与NULL值有关。以下脚本确保如果存在某些列为NULL的行,则匹配它们。

--Create the sample source table.
--Notice that the columns allow NULL values...
CREATE TABLE #T1
(
  Col1 nvarchar(10) NULL,
  Col2 nvarchar(10) NULL,
  Col3 nvarchar(10) NULL
)

--Create the sample destination table.
CREATE TABLE #T2
(
  Col1 nvarchar(10) NULL,
  Col2 nvarchar(10) NULL,
  Col3 nvarchar(10) NULL
)
GO
--Populate the source table with some initial rows
--Notice that some column values are NULL
INSERT INTO #T1 (Col1,Col2,Col3) 
VALUES ('A','B',NULL),
        ('D','E','F'),    
        ('G',NULL,'I');
GO
--Verify the data in the tables 
--Table #T1 should have data in it
SELECT * FROM #T1;
--Table #T2 should be empty
SELECT * FROM #T2;
GO
--Copy the rows from #T1 to #T2
--where there are no matching rows.
INSERT INTO #T2(Col1,Col2,Col3)
SELECT 
  T1.Col1,T1.Col2,T1.Col3 
FROM #T1 AS T1
LEFT OUTER JOIN #T2 AS T2
ON  IsNull(T1.Col1,'') = IsNull(T2.Col1,'')
AND IsNull(T1.Col2,'') = IsNull(T2.Col2,'')
AND IsNull(T1.Col3,'') = IsNull(T2.Col3,'')
WHERE T2.Col1 IS NULL AND T2.Col2 IS NULL AND T2.Col3 IS NULL
GO
--Verify the data in the tables.  They should be the same now.
SELECT * FROM #T1;
SELECT * FROM #T2;
GO
--Add a some new rows to #T1
INSERT INTO #T1 (Col1,Col2,Col3) 
VALUES ('J','K','L'),
       ('M',NULL,'O'),
       ('P','Q','R'),
       ('A','B','C'); --THIS LAST ROW DOES NOT MATCH THE EXISTING A,B,NULL ROW AND WILL BE INSERTED ON MERGE
GO
--Merge the tables, and yes you could use MERGE where the ON clause handled NULLs
INSERT INTO #T2(Col1,Col2,Col3)
SELECT 
  T1.Col1,T1.Col2,T1.Col3 
FROM #T1 AS T1
LEFT OUTER JOIN #T2 AS T2
ON  IsNull(T1.Col1,'') = IsNull(T2.Col1,'')
AND IsNull(T1.Col2,'') = IsNull(T2.Col2,'')
AND IsNull(T1.Col3,'') = IsNull(T2.Col3,'')
WHERE T2.Col1 IS NULL AND T2.Col2 IS NULL AND T2.Col3 IS NULL
GO
--Verify the new data is in both tables.
SELECT * FROM #T1;
SELECT * FROM #T2;
GO
--Cleanup
DROP TABLE #T1;
DROP TABLE #T2;