Question

当我在执行涉及插入数据的代码时，或者如果存在，则从文件更新时，我遇到了一个大问题。我正在使用实体框架。文件接近16 000行

我试图插入它们，但我花了很多时间做这个：超过5小时的x文件。

我决定改变它，但我不知道如何开始。

我读了一些关于保存每个1000,5000,10000行的内容，但是当我尝试这样做时，我得到了异常。

Store update, insert, or delete statement affected an unexpected number of rows (0).
Entities may have been modified or deleted since entities were loaded. Refresh
ObjectStateManager entries.

当我尝试修复它时，我遇到了这个异常

AcceptChanges can not continue because the object's key values conflict with another
object in ObjectStateManager. Make sure that the key values are unique before calling 
AcceptChanges.

这是我的代码

//before this I passed db data to lists
foreach (var record in records)
{
    if (record != null)
    {
        try
        {
            if (!_location.Any(s => s.Id.Trim() == record.Id))
            {               
                Location l = new Location
                    {
                        Id = record.Id,
                        Name = record.Name,
                        Address = record.Address,
                        City = record.City,
                        State = record.State,
                        Zip = record.Zip.ToString(),
                        Zip2 = record.Zip2,
                        Custom1 = record.Custom1,                       
                        CreatedDate = date,
                        UpdatedDate = date
                    };

                db.location.Add(l);  
                db.SaveChanges();                              
                _location.Add(l);                                        
            }
            else
            {                                
                if (!_sales.Any(s => s.LocationId == record.Id && s.ReportDate == record.ReportDate))
                {
                    Sale s = new Sale
                    {
                        ReportDate = record.ReportDate,
                        CreatedDate = date,
                        UpdatedDate = date,
                        Amount = record.Amount,
                        LocationId = record.Id,
                    };
                    db.Sale.Add(s);  
                    db.SaveChanges();                  
                    _sales.Add(s);
                }
                else
                {

                    if (!_sales.Any(s => s.LocationId == record.Id && s.ReportDate == record.ReportDate && s.Amount == record.Amount))
                    {                       
                        if ((_sales.Where(s => s.LocationId == record.Id && s.ReportDate == record.ReportDate)).Count() == 1)
                        {                           
                            var sale = _sales.SingleOrDefault(s => s.LocationId == record.Id && s.ReportDate == record.ReportDate);
                            sale.UpdatedDate = date;
                            sale.Amount = record.Amount;
                            db.Entry(sale).State = EntityState.Modified;  
                            db.SaveChanges();                                                                   
                        }                                                
                    }
                }                                        
            }                        
        }
        catch (Exception ex)
        {

        }

    }
}

我所做的改变是：

在foreach结束时只更改所有db.SaveChanges，我得到了First异常（当我将它们分组为3000时）

更改实体状态，我得到了第二个例外。

我很感激你的帮助，

感谢。

Answer 1

据我了解，您希望将数据从文件导入数据。如果我是对的，我面临同样的问题，但每次更新有超过20,000件物品。经过一些测试后，我找到了以下解决方案，可以解决您的问题：

对于海量数据导入，EF的速度很慢
所以我使用SqlBulkCopy-Command在额外的导入表中导入数据，并通过存储过程在数据库中导入导入的数据。导入数据的方法可能如下所示：


private void DoMassDataImport(DataTable tab)
 {            
    var conn = new SqlConnection(this.sqlConnectionString);
    try
    {
        if (conn.State != ConnectionState.Open)
        {
            conn.Open();
        }

        SqlTransaction transaction = conn.BeginTransaction();

        using (SqlBulkCopy copy = new SqlBulkCopy(conn, SqlBulkCopyOptions.Default, transaction))
        {
            copy.BulkCopyTimeout = 10000;
            copy.DestinationTableName = "dbo.TargetTable";
            copy.WriteToServer(tab);
        }
        transaction.Commit();
        result.Success = true;                
    }
    catch (Exception ex)
    {
        result.Success = false;
        this.LogEvent(ex.Message + Environment.NewLine + ex.StackTrace);
    }
    finally
    {
        conn.Close();                
    }
}

可以从代码中调用存储过程，如果处理时间不超过1分钟就可以使用EF，但如果需要更长时间，则应考虑使用这样的纯SQL命令：


private void ProcessImportedData()
{
    SqlConnection connection = null;    
    try
    {
        connection = new SqlConnection(this.sqlConnectionString);
        if (connection.State != ConnectionState.Open)
        {
            connection.Open();
        }

        SqlTransaction transaction = connection.BeginTransaction();
        using (SqlCommand processingCommand = connection.CreateCommand())
        {
            processingCommand.Transaction = transaction;
            processingCommand.CommandTimeout = 10000;
            processingCommand.CommandText = "dbo.StoredProcedure ";
            processingCommand.CommandType = CommandType.StoredProcedure;
            processingCommand.ExecuteNonQuery();            
            result.Success = true
        }

        transaction.Commit();
    }
    catch (Exception ex)
    {
        result.Success = false;     
    }
    finally
    {
        if (connection != null)
        {
            connection.Close();

        }
    }
}

使用此方法将数据导入的时间从超过5小时减少到不到5分钟

使用ef多个实体保存或更新数据

1 个答案: