Question

我在C＃批处理作业中使用下面的代码来处理大量（20k +）的更新和插入。但是，在测试期间，我可以看到，如果存在问题，例如违反约束，我将只返回第一条错误消息，并且没有关于导致问题的记录（或记录）的信息。

是否可以使用可以为我们提供此功能的.Net或T-SQL来实现错误处理方法或技术？

C＃

    private static string insertCommand =
        "INSERT (ChannelCode, DrmTerrDesc, IndDistrnId, StateCode, ZipCode, EndDate, EffectiveDate, LastUpdateId, LastUpdateDate, ErrorCodes, Status) " +
        "VALUES(Source.ChannelCode, Source.DrmTerrDesc, Source.IndDistrnId, Source.StateCode, Source.ZipCode, Source.EndDate, Source.EffectiveDate, Source.LastUpdateId, Source.LastUpdateDate, Source.ErrorCOdes, Source.Status)";

    private static string updateCommand = "UPDATE SET Target.ChannelCode = Source.ChannelCode, Target.DrmTerrDesc = Source.DrmTerrDesc, Target.IndDistrnId = Source.IndDistrnId," +
                                                "Target.StateCode = Source.StateCode, Target.ZipCode = Source.ZipCode, Target.EndDate = Source.EndDate, Target.EffectiveDate = Source.EffectiveDate," +
                                                "Target.LastUpdateId = Source.LastUpdateId, Target.LastUpdateDate = Source.LastUpdateDate, Target.ErrorCodes = Source.ErrorCodes," +
                                                "Target.Status = Source.Status ";

    public static int Update(List<ZipCodeTerritory> updates, Dictionary<object, string> errorList)
    {
        int results = 0;
        try
        {
            //Load updates into datatable
            DataTable table = LoadData(updates, true);

            //Script to create temp table
            string tmpTable =   "CREATE TABLE [dbo].[ZipCodeTerritoryTemp]( " +
                                "[ChannelCode] [char](1) NOT NULL, " +
                                "[DrmTerrDesc] [nvarchar](30) NOT NULL, " +
                                "[IndDistrnId] [char](3) NULL, " +
                                "[StateCode] [char](3) NOT NULL, " +
                                "[ZipCode] [char](9) NULL, " +
                                "[EndDate] [date] NOT NULL, " +
                                "[EffectiveDate] [date] NOT NULL, " +
                                "[LastUpdateId] [char](8) NULL, " +
                                "[LastUpdateDate] [date] NULL, " +
                                "[Id] [int] NULL, " +               
                                "[ErrorCodes] [varchar](255) NULL, " +
                                "[Status] [char](1) NULL)";

            using (SqlConnection connection = new SqlConnection(connString))
            {
                connection.Open();

                //Create temp table
                SqlCommand cmd = new SqlCommand(tmpTable, connection);
                cmd.ExecuteNonQuery();

                try
                {

                    using (SqlBulkCopy bulkCopy = new SqlBulkCopy(connection))
                    {
                        //Write to temp table
                        bulkCopy.DestinationTableName = "ZipCodeTerritoryTemp";
                        bulkCopy.WriteToServer(table);

                        //Merge changes in temp table with ZipCodeTerritory
                        string mergeSql = "merge ZipCodeTerritory as Target " +
                                          "using ZipCodeTerritoryTemp as Source " +
                                          "on " +
                                          "Target.Id = Source.Id " +
                                          "when matched then " +
                                          updateCommand +
                                          "when not matched then " +
                                          insertCommand + ";";

                        cmd.CommandText = mergeSql;
                        results = cmd.ExecuteNonQuery();
                    }
                }
                catch (Exception ex)
                {
                    SendEmail.ErrorMail(ex.Message);
                }
                finally
                {
                    //Drop temp table
                    SqlCommand final = new SqlCommand("DROP TABLE [dbo].[ZipCodeTerritoryTemp]", connection);
                    final.ExecuteNonQuery();
                }
            }
        }
        catch (Exception ex)
        {
            SendEmail.ErrorMail(ex.Message);
        }
        return results;
    }

Answer 1

简短的回答是，您无法通过执行MERGE语句来确定这一点，您必须在执行MERGE之前检查这些冲突。

换句话说（我不能强调这一点）： 始终验证您的输入。

有两点可以进行验证：在将数据批量复制到临时表之前以及将临时表合并到目标之前。根据数据问题的性质，您可以在数据到达服务器之前进行大量验证。

MERGE语句通常会遇到三类主要问题：

键冲突（源中的重复行）
数据格式错误（例如，无法正确翻译为DATE的日期的字符串表示）
约束失败（在不允许的情况下为空，外键错误等）

在将数据推送到服务器之前，通常可以检测前两个。第三个取决于约束的性质......但通常我们可以在它们到达服务器之前解决这些问题。

可以通过按键对数据进行分组来检测数据中的密钥冲突（在本例中为Id）。我们假设您有两条具有相同Id值的记录，但您希望合并到具有最高LastUpdateDate的记录中。一个选择是：

var cleanupdates = 
    from update in updates
    group update by update.Id into grp
    select grp.OrderByDescending(u => u.LastUpdateDate).First();

如果约束问题与空值相关，请使用where子句过滤掉那些具有无效空值的记录。如果它们与外键约束相关，请将这些键加载到列表中并对其进行过滤。您可以使用LINQ查询覆盖大量验证。

重点是你做验证。否则，MERGE会失败，你不会知道原因。

Answer 2

对于您的合并声明，您缺少一件小事

merge ZipCodeTerritory as Target " +
"using (SELECT * FROM ZipCodeTerritoryTemp) as Source " + --<-- need to select data
"on " +
"Target.Id = Source.Id " +
"when matched then " +
updateCommand +
"when not matched then " +
insertCommand + ";";

但在你继续采用这种方法之前，来自Aaron Bertrand的文章 Read This 关于MERGE声明的问题。

我建议您使用IF EXISTS这样的方法......

<强>更新

UPDATE T
SET  T.Col1 = S.Col1,
     T.Col2 = S.Col2,
     T.Col3 = S.Col3
FROM ZipCodeTerritory T INNER JOIN ZipCodeTerritoryTemp S
ON   T.id = S.id

插入

INSERT INTO ZipCodeTerritory(Col1, Col2, Col3, ....) SELECT S.Col1, S.Col2, S.Col3, .... FROM ZipCodeTerritoryTemp S LEFT JOIN ZipCodeTerritory T ON S.ID = T.ID WHERE T.ID IS NULL

如何识别在MERGE语句期间导致错误的行/行以及原因

2 个答案: