插入关系非常慢

时间:2016-11-22 08:16:35

标签: c# entity-framework

我必须插入多个关系并且遇到Context.SaveChanges操作的问题,这需要永远完成。我已经尝试了多种方法将这些实体添加到数据库中,但似乎没有任何帮助我。

我的模型以以下方式构建:

public class Agreement : GdSoftDeleteEntity
{
    public DateTime Date { get; set; }
    public AgreementType AgreementType { get; set; }

    public virtual ICollection<PersonAgreementRelation> PersonAgreementRelations { get; set; }
    public virtual ICollection<ImageSearchAppointment> ImageSearchAppointments { get; set; }
}

public class Person : GdSoftDeleteEntity
{
    public string Name { get; set; }
    public string FirstName { get; set; }

    // E-mail is in identityuser
    //public string EmailAddress { get; set; }

    public virtual PersonType PersonType { get; set; }

    public virtual ICollection<PersonAgreementRelation> PersonAgreementRelations { get; set; }
    public virtual ICollection<PersonPersonRelation> PersonMasters { get; set; }
    public virtual ICollection<PersonPersonRelation> PersonSlaves { get; set; }
}

public class PersonAgreementRelation : GdSoftDeleteEntity
{
    public int PersonId { get; set; }
    public virtual Person Person { get; set; }

    public int AgreementId { get; set; }
    public virtual Agreement Agreement { get; set; }

    public virtual PersonAgreementRole PersonAgreementRole { get; set; }
}

public class ImageSearchAppointment : GdSoftDeleteEntity
{
    public string Name { get; set; }
    public bool ShowResultsToCustomer { get; set; }
    public bool HasImageFeed { get; set; }
    public int AgreementId { get; set; }
    public virtual Agreement Agreement { get; set; }

    public Periodicity Periodicity { get; set; }
    public PeriodicityCategory PeriodicityCategory { get; set; }

    public virtual ICollection<ImageSearchCommand> ImageSearchCommands { get; set; }
    public virtual ICollection<ImageSearchAppointmentWebDomainWhitelist> ImageSearchAppointmentWebDomainWhitelists { get; set; }
    public virtual ICollection<ImageSearchAppointmentWebDomainExtension> ImageSearchAppointmentWebDomainExtensions { get; set; }
}

public class ImageSearchCommand : GdSoftDeleteEntity
{
    public int ImageSearchAppointmentId { get; set; }
    public virtual ImageSearchAppointment ImageSearchAppointment { get; set; }

    public int? ImageSearchAppointmentCredentialsId { get; set; }
    public virtual ImageSearchAppointmentCredentials ImageSearchAppointmentCredentials { get; set; }

    public DateTime Date { get; set; }

    //public bool Invoiced { get; set; }
    public int NumberOfImages { get; set; }
    public DateTime ImageCollectionProcessedDate { get; set; }

    public virtual ICollection<ImageSearchExecution> ImageSearchExecutions { get; set; }
}

在我的服务中,我写了以下代码:

public int AddAgreement(int personId, AgreementDto agreementDto)
        {
            Context.Configuration.LazyLoadingEnabled = false;
            //var person = Context.Persons.SingleOrDefault(el => el.Id == personId);
            var person = Context.Persons
                .SingleOrDefault(x => x.Id == personId);
            if (person == null)
            {
                throw new GraphicsDetectiveInvalidDataTypeException($"No person found for Id: {personId}");
            }

            if (agreementDto == null)
            {
                throw new GraphicsDetectiveInvalidDataTypeException("Invalid agreementDto");
            }

            //TODO: Check if OKAY!!!

            if (agreementDto.ImageSearchAppointmentDto.Count == 0)
            {
                throw new GraphicsDetectiveInvalidDataTypeException("Count of imagesearchappointments can't be lower than 0");
            }

            //set agreement properties
            var agreement = new Agreement
            {
                Date = agreementDto.DateTime,
                AgreementType = AgreementType.WwwImageSearch,
                //ImageSearchAppointments = new List<ImageSearchAppointment>(),
                //IsDeleted = false
            };
            Context.Agreements.Add(agreement);
            Context.SaveChanges();

            //var personAdminId = Context.Users.Single(x => x.Email == ConfigurationManager.AppSettings["DefaultGdAdminEmail"]).PersonId;
            // Dit werkt niet. Moet in 2 stappen
            //set personagreementrelations for new agreement
            var adminEmail = ConfigurationManager.AppSettings["DefaultGdAdminEmail"];    
            var personAdminId = Context.Users
                .SingleOrDefault(x => x.Email == adminEmail)
                .PersonId;

            var personPmId = Context.Persons.Single(x => x.Name == "My name").Id;
            var personAgreementRelations = new List<PersonAgreementRelation>()
                {
                    new PersonAgreementRelation
                    {
                        AgreementId = agreement.Id,
                        PersonId = personId,
                        PersonAgreementRole = PersonAgreementRole.Client,
                    },
                    new PersonAgreementRelation
                    {
                        AgreementId = agreement.Id,
                        PersonAgreementRole = PersonAgreementRole.Supplier,
                        PersonId = personPmId,
                    },
                     new PersonAgreementRelation
                    {
                        AgreementId = agreement.Id,
                        PersonAgreementRole = PersonAgreementRole.Admin,
                        PersonId = personAdminId,
                    }
                };
            foreach (var personAgreementRelation in personAgreementRelations)
            {
                Context.PersonAgreementRelations.Add(personAgreementRelation);
            }

            Context.Configuration.ValidateOnSaveEnabled = false;
            Context.Configuration.AutoDetectChangesEnabled = false;

            Context.SaveChanges();

            Context.Configuration.ValidateOnSaveEnabled = true;
            Context.Configuration.AutoDetectChangesEnabled = true;
            Context.Configuration.LazyLoadingEnabled = true;

            return agreement.Id;
        }

        public void AddFirstImageSearchAppointmentToAgreement(int agreementId, ImageSearchAppointmentDto imageSearchAppointmentDto)
        {
            Context.Configuration.LazyLoadingEnabled = false;
            var agreement = Context.Agreements.SingleOrDefault(x => x.Id == agreementId);
            if (agreement == null)
            {
                throw new GraphicsDetectiveInvalidDataTypeException($"No agreement found for id {agreementId}");
            }
            var appointmentType = imageSearchAppointmentDto;
            if (appointmentType == null)
            {
                throw new GraphicsDetectiveInvalidDataTypeException($"No valid imageSearchAppointment");
            }
            if (appointmentType.ImageSearchCommandDto.Count == 0)
            {
                throw new GraphicsDetectiveInvalidDataTypeException("No imageSearchCommand");
            }

            var imageSearchAppointment = new ImageSearchAppointment
            {
                AgreementId = agreement.Id,
                Agreement = agreement,
                Name = appointmentType.Name,
                Periodicity = appointmentType.Periodicity,
                PeriodicityCategory = appointmentType.PeriodicityCategory,
                ShowResultsToCustomer = appointmentType.ShowResultsToCustomer,
                ImageSearchAppointmentWebDomainExtensions = new List<ImageSearchAppointmentWebDomainExtension>(),
                ImageSearchCommands = new List<ImageSearchCommand>(),
                ImageSearchAppointmentWebDomainWhitelists = new List<ImageSearchAppointmentWebDomainWhitelist>(),
                IsDeleted = false
            };

            var imageSearchCommandDto = appointmentType.ImageSearchCommandDto.Single();
            var imageSearchCommand = new ImageSearchCommand()
            {
                ImageSearchAppointment = imageSearchAppointment,
                Date = imageSearchCommandDto.Date,
                NumberOfImages = imageSearchCommandDto.NumberOfImages,
                ImageCollectionProcessedDate = imageSearchCommandDto.ImageCollectionProcessedDate,
                IsDeleted = false
            };

            if (imageSearchCommandDto.ImageSearchAppointmentCredentialsDto != null)
            {
                imageSearchCommand.ImageSearchAppointmentCredentials = new ImageSearchAppointmentCredentials
                {
                    FtpProfileType = imageSearchCommandDto.ImageSearchAppointmentCredentialsDto.FtpProfileType,
                    Location = imageSearchCommandDto.ImageSearchAppointmentCredentialsDto.Location,
                    Username = imageSearchCommandDto.ImageSearchAppointmentCredentialsDto.Username,
                    Password = imageSearchCommandDto.ImageSearchAppointmentCredentialsDto.Password,
                    UsePassive = imageSearchCommandDto.ImageSearchAppointmentCredentialsDto.UsePassive,
                    IsDeleted = false
                };
            }
            imageSearchAppointment.ImageSearchCommands.Add(imageSearchCommand);

            if (!imageSearchAppointment.ShowResultsToCustomer)
            {
                var webDomainExtensions = appointmentType.WebDomainExtensionDtos
                    .Select(x => new ImageSearchAppointmentWebDomainExtension()
                    {
                        ImageSearchAppointment = imageSearchAppointment,
                        WebDomainExtensionId = x.Id
                    })
                    .ToList();

                imageSearchAppointment.ImageSearchAppointmentWebDomainExtensions = webDomainExtensions;
            }

            Context.ImageSearchAppointments.Add(imageSearchAppointment);
            Context.SaveChanges();

            Context.Configuration.LazyLoadingEnabled = true;
        }

我使用dotTrace来分析这些功能,将新实体添加到我的数据库大约需要9分钟。

数据库是Azure SQL数据库,层S3

我尝试了所提出的解决方案并调整了我的代码如下:

public int AddAgreement(int personId, AgreementDto agreementDto)
        {
            var agreementId = 0;
            using (var context = new GdDbContext())
            {
                GdDbConfiguration.SuspendExecutionStrategy = true;
                context.Configuration.LazyLoadingEnabled = true;
                //var person = Context.Persons.SingleOrDefault(el => el.Id == personId);
                var person = context.Persons
                    .SingleOrDefault(x => x.Id == personId);
                if (person == null)
                {
                    throw new GraphicsDetectiveInvalidDataTypeException($"No person found for Id: {personId}");
                }

                //var personAdminId = Context.Users.Single(x => x.Email == ConfigurationManager.AppSettings["DefaultGdAdminEmail"]).PersonId;
                // Dit werkt niet. Moet in 2 stappen
                //set personagreementrelations for new agreement
                var adminEmail = ConfigurationManager.AppSettings["DefaultGdAdminEmail"];
                var personAdminId = context.Users
                    .Where(x => x.Email == adminEmail)
                    .Include(x => x.Person)
                    .First()
                    .Person.Id;

                var personPmId = context.Persons.First(x => x.Name == "My name").Id;

                using (var dbContextTransaction = context.Database.BeginTransaction())
                {
                    try
                    {

                        if (agreementDto == null)
                        {
                            throw new GraphicsDetectiveInvalidDataTypeException("Invalid agreementDto");
                        }

                        //TODO: Check if OKAY!!!

                        if (agreementDto.ImageSearchAppointmentDto.Count == 0)
                        {
                            throw new GraphicsDetectiveInvalidDataTypeException("Count of imagesearchappointments can't be lower than 0");
                        }

                        //set agreement properties
                        var agreement = new Agreement
                        {
                            Date = agreementDto.DateTime,
                            AgreementType = AgreementType.WwwImageSearch,
                            //ImageSearchAppointments = new List<ImageSearchAppointment>(),
                            //IsDeleted = false
                        };
                        context.Agreements.Add(agreement);
                        //Context.SaveChanges();

                        var personAgreementRelations = new List<PersonAgreementRelation>()
                        {
                            new PersonAgreementRelation
                            {
                                //Agreement = agreement,
                                AgreementId = agreement.Id,
                                PersonId = personId,
                                //Person = person,
                                PersonAgreementRole = PersonAgreementRole.Client,
                                //IsDeleted = false
                            },
                            new PersonAgreementRelation
                            {
                                //Agreement = agreement,
                                AgreementId = agreement.Id,
                                PersonAgreementRole = PersonAgreementRole.Supplier,
                                PersonId = personPmId,
                                //Person = personPm,
                                //IsDeleted = false
                            },
                             new PersonAgreementRelation
                            {
                                //Agreement = agreement,
                                AgreementId = agreement.Id,
                                PersonAgreementRole = PersonAgreementRole.Admin,
                                PersonId = personAdminId,
                                //Person = personAdmin,
                            }
                        };

                        foreach (var personAgreementRelation in personAgreementRelations)
                        {
                            context.PersonAgreementRelations.Add(personAgreementRelation);
                        }
                        //agreement.PersonAgreementRelations = personAgreementRelations;

                        //Context.Agreements.Add(agreement);

                        context.Configuration.ValidateOnSaveEnabled = false;
                        context.Configuration.AutoDetectChangesEnabled = false;

                        //await Context.SaveChangesAsync();

                        context.SaveChanges();
                        dbContextTransaction.Commit();
                        //await Task.Run(async () => await Context.SaveChangesAsync());

                        context.Configuration.ValidateOnSaveEnabled = true;
                        context.Configuration.AutoDetectChangesEnabled = true;
                        context.Configuration.LazyLoadingEnabled = false;

                        agreementId = agreement.Id;
                    }
                    catch (Exception ex)
                    {
                        dbContextTransaction.Rollback();
                        throw ex;
                    }
                }
                GdDbConfiguration.SuspendExecutionStrategy = false;
            }

            return agreementId;
        }

但它花费的时间和以前一样多

2 个答案:

答案 0 :(得分:0)

您可以按照以下提到的建议来改善上述方法的效果。

  1. 使用FirstOrDefault()代替SingleOrDefault()FirstOrDefault()是最快的方法。

  2. 我可以看到你在同一个方法上使用Context.SaveChanges()方法次数。这会降低方法的执行力。所以你必须避免这种情况。而不是使用事务。

  3. 像这样:EF Transactions

    using (var context = new YourContext()) 
                { 
                    using (var dbContextTransaction = context.Database.BeginTransaction()) 
                    { 
                        try 
                        { 
                            // your operations here
    
                            context.SaveChanges(); //this called only once
                            dbContextTransaction.Commit(); 
                        } 
                        catch (Exception) 
                        { 
                            dbContextTransaction.Rollback(); 
                        } 
                    } 
                } 
    
    1. 如果上面的内容不能给出足够的改进,你可以考虑存储过程的实现。

答案 1 :(得分:0)

您的代码存在一些性能问题

添加效果

foreach (var personAgreementRelation in personAgreementRelations)
{
    Context.PersonAgreementRelations.Add(personAgreementRelation);
}


Context.Configuration.ValidateOnSaveEnabled = false;
Context.Configuration.AutoDetectChangesEnabled = false;

添加多个实体,然后禁用AutoDetectChanges。你通常做反向

根据您上下文中的实体数量,可能会严重影响您的效果

在方法“AddFirstImageSearchAppointmentToAgreement”中,似乎你使用的外部上下文如果已包含数千个实体则可能非常糟糕。

请参阅:Improve Entity Framework Add Performance

使用不当,使用Add方法向上下文添加实体比在数据库中保存更多时间!

SaveChanges与批量插入与BulkSaveChanges的对比

SaveChanges非常慢。对于要保存的每个记录,都需要数据库往返。对于SQL Azure用户来说尤其如此,因为存在额外的延迟。

某些库允许您执行批量插入

请参阅:

免责声明:我是该项目的所有者Entity Framework Extensions

此库具有BulkSaveChanges功能。它的工作原理与SaveChanges完全相同,但 WAY FASTER

// Context.SaveChanges();
Context.BulkSaveChanges();

编辑:添加其他信息#1

  

我在Pastebin中粘贴了我的新代码:link

<强>交易

为什么在选择数据并将实体添加到上下文时启动事务?这只是一个非常糟糕的交易使用。

必须尽可能晚地启动交易。因为BulkSaveChanges已经在事务中执行,所以没有必要创建它。

<强> Async.Result

 var personAdminId = context.Users.FirstOrDefaultAsync(x => x.Email == adminEmail).Result.PersonId;

我不明白为什么你在这里使用异步方法......

  • 在最好的情况下,您获得与使用非异步方法类似的性能
  • 在更糟糕的情况下,您会遇到一些performance issue使用异步方法

缓存条目

var adminEmail = ConfigurationManager.AppSettings["DefaultGdAdminEmail"];
var personAdminId = context.Users.FirstOrDefaultAsync(x => x.Email == adminEmail).Result.PersonId;

我不知道你多少次调用AddAgreement方法,但我怀疑管理员会改变。

因此,如果您将其调用10,000次,则每次进行10,000次数据库往返以获得相同的精确值。

改为创建一个静态变量,只获取一次值!你肯定会在这里节省很多时间

以下是我通常处理这种静态变量的方法:

var personAdminId = My.UserAdmin.Id;

public static class My
{
    private static User _userAdmin;
    public static User UserAdmin
    {
        get
        {
            if (_userAdmin == null)
            {
                using (var context = new GdDbContext())
                {
                    var adminEmail = ConfigurationManager.AppSettings["DefaultGdAdminEmail"];
                    _userAdmin = context.Users.FirstOrDefault(x => x.Email == adminEmail);
                }
            }

            return _userAdmin;
        }
    }
}

<强> LazyLoadingEnabled

在第一个代码中,您将LazyLoadingEnabled设置为false,但不在您的Pastebin代码中,

禁用LazyLoading可能会有所帮助,因为它不会创建代理实例。

取10米而不是9米

删除事务后请告诉我,如果性能稍好一点,请再次禁用LazyLoading。

下一步将是了解一些统计数据:

  • 大约调用AddAgreement方法的次数
  • 您的数据库中有多少人
  • 通过AddAgreement方法保存平均实体数量

编辑:添加其他信息#2

目前,提高性能的唯一方法是减少数据库往返次数。

我发现你每次都在搜索 personAdminId 。通过将此值缓存到某个静态变量,您可以在此处节省大约30秒到1分钟。

你仍然没有回答这三个问题:

  • 大约调用AddAgreement方法的次数
  • 您的数据库中有多少人
  • 通过AddAgreement方法保存平均实体数量

这些问题的目标是了解什么是缓慢的!

例如,如果您调用AddAgreement方法10,000次并且数据库中只有2000个人,那么您可能最好在两个字典中缓存2000个人以节省20,000个数据库往返(节省一到两分钟? )。