大批量插入会导致事务中止或超时

时间:2015-04-12 14:19:38

标签: c# sql sql-server entity-framework bulkinsert

问题

当在两个数据库上进行数十万行的批量插入(以编程方式,相同的实例,相同的模式)和每个数据库大约17个表时,我几乎总是收到Transcaction Aborted或Timeout Expired异常。

如何

使用EntityFramework 6数据上下文,我在数据库上运行SqlCommand。

using (var tran = new TransactionScope(TransactionScopeOption.Required, transactionOptions))
{
    context.Database.ExecuteSqlCommand(
        "BULK INSERT " + table +
        " FROM '" + Directory +
        table + ".csv' WITH ( DATAFILETYPE = 'widechar', FIRSTROW = 2, FIELDTERMINATOR = '[TERM]', ERRORFILE = '" +
        Logfile + DateTime.Now.ToString(@"yyyy-MM-dd") + "-R" + DateTime.Now.Second + DateTime.Now.Millisecond +
        ".txt', ROWTERMINATOR = '\n', KEEPNULLS, TABLOCK )"); 

    tran.Complete();
}

设置超时值

为了解决我面临的问题,我已将命令和事务超时设置为18分钟。在十八分钟过去之前很久就会发生异常。

context.Database.CommandTimeout = 1080;
TransactionOptions transactionOptions = new TransactionOptions();
transactionOptions.Timeout = TimeSpan.FromMinutes(18);

完全例外

System.Data.SqlClient.SqlException (0x80131904): Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding. ---> System.ComponentModel.Win32Exception (0x80004005): The wait operation timed out
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlCommand.RunExecuteNonQueryTds(String methodName, Boolean async, Int32 timeout, Boolean asyncWrite)
   at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean asyncWrite)
   at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
   at System.Data.Entity.Infrastructure.Interception.DbCommandDispatcher.<NonQuery>b__0(DbCommand t, DbCommandInterceptionContext`1 c)
   at System.Data.Entity.Infrastructure.Interception.InternalDispatcher`1.Dispatch[TTarget,TInterceptionContext,TResult](TTarget target, Func`3 operation, TInterceptionContext interceptionContext, Action`3 executing, Action`3 executed)
   at System.Data.Entity.Infrastructure.Interception.DbCommandDispatcher.NonQuery(DbCommand command, DbCommandInterceptionContext interceptionContext)
   at System.Data.Entity.Internal.InterceptableDbCommand.ExecuteNonQuery()
   at System.Data.Entity.Core.Objects.ObjectContext.<>c__DisplayClass59.<ExecuteStoreCommand>b__58()
   at System.Data.Entity.Core.Objects.ObjectContext.ExecuteInTransaction[T](Func`1 func, IDbExecutionStrategy executionStrategy, Boolean startLocalTransaction, Boolean releaseConnectionOnSuccess)
   at System.Data.Entity.Core.Objects.ObjectContext.<>c__DisplayClass59.<ExecuteStoreCommand>b__57()
   at System.Data.Entity.SqlServer.DefaultSqlExecutionStrategy.Execute[TResult](Func`1 operation)
   at System.Data.Entity.Core.Objects.ObjectContext.ExecuteStoreCommand(TransactionalBehavior transactionalBehavior, String commandText, Object[] parameters)
   at System.Data.Entity.Internal.InternalContext.ExecuteSqlCommand(TransactionalBehavior transactionalBehavior, String sql, Object[] parameters)
   at System.Data.Entity.Database.ExecuteSqlCommand(TransactionalBehavior transactionalBehavior, String sql, Object[] parameters)
   at System.Data.Entity.Database.ExecuteSqlCommand(String sql, Object[] parameters)
   at MyProject.Updaters.MSSQL.Csv.ImportCsvFiles() in c:\Users\MyUser\Desktop\MyProject2\XMLtoDBcron\Updaters\MSSQL\Csv.cs:line 48
ClientConnectionId:fef47d9c-3e13-444f-939b-f9bd570abb36

+

System.Transactions.TransactionAbortedException: The transaction has aborted. ---> System.TimeoutException: Transaction Timeout
   --- End of inner exception stack trace ---
   at System.Transactions.TransactionStateAborted.BeginCommit(InternalTransaction tx, Boolean asyncCommit, AsyncCallback asyncCallback, Object asyncState)
   at System.Transactions.CommittableTransaction.Commit()
   at System.Transactions.TransactionScope.InternalDispose()
   at System.Transactions.TransactionScope.Dispose()
   at MyProject.Updaters.MSSQL.Csv.InsertCsvData() in c:\Users\MyUser\Desktop\MyProject2\XMLtoDBcron\Updaters\MSSQL\Csv.cs:line 109
   at MyProject.Updaters.MSSQL.Csv.ImportCsvFiles() in c:\Users\MyUser\Desktop\MyProject2\XMLtoDBcron\Updaters\MSSQL\Csv.cs:line 74

怎么做

我无法在SQL Server中找到任何设置或类似内容(使用SSMS)。您将要求提供任何有助于提供答案的额外数据。

1 个答案:

答案 0 :(得分:1)

关于sql命令执行期间的timout,我建议你通过包含在DbContext中的ObjectContext设置CommandTimeout:

((IObjectContextAdapter)context).ObjectContext.CommandTimeout = 1080;

这种方法对我来说一直很有用。