为什么NHibernate拒绝批量插入?

时间:2010-08-04 14:58:31

标签: nhibernate sql-server-2008 bulkinsert batching

对SQL Server 2008使用NHibernate 2.1.2.4000。目标表没有触发器或无关索引。它很简单:

create table LogEntries (
    Id INT IDENTITY NOT NULL,
   HostName NVARCHAR(32) not null,
   UserName NVARCHAR(64) not null,
   LogName NVARCHAR(512) not null,
   Timestamp DATETIME not null,
   Level INT not null,
   Thread NVARCHAR(64) not null,
   Message NVARCHAR(MAX) not null,
   primary key (Id)
)

我的实体映射是:

<class name="LogEntry" table="LogEntries">
    <id name="Id" unsaved-value="0">
        <generator class="native"/>
    </id>
    <property name="HostName" length="32" not-null="true"/>
    <property name="UserName" length="64" not-null="true"/>
    <property name="LogName" length="512" not-null="true"/>
    <property name="Timestamp" type="utcdatetime" not-null="true"/>
    <property name="Level" not-null="true"/>
    <property name="Thread" length="64" not-null="true"/>
    <property name="Message">
        <column name="Message" sql-type="NVARCHAR(MAX)" not-null="true"/>
    </property>
</class>

现在,请考虑以下测试用例:

[Fact]
public void bulk_insert_test()
{
    var batchSize = 100;
    var numberItems = 10000;

    var configuration = new NHibernate.Cfg.Configuration().Configure();
    configuration.SetProperty("connection.connection_string", @"my_conn_string");
    configuration.SetProperty("adonet.batch_size", batchSize.ToString());
    var sessionFactory = configuration.BuildSessionFactory();

    var ts = this.WriteWithNH(sessionFactory, numberItems);
    ////var ts = this.WriteWithBC(sessionFactory, numberItems, batchSize);

    Console.WriteLine("Saving {0} items with batch size {1}: {2}", numberItems, batchSize, ts);
}

public TimeSpan WriteWithNH(ISessionFactory sessionFactory, int numberItems)
{
    using (var session = sessionFactory.OpenStatelessSession())
    using (var transaction = session.BeginTransaction())
    {
        session.Insert(new LogEntry()
        {
            HostName = "host",
            UserName = "user",
            LogName = "log",
            Level = 0,
            Thread = "thread",
            Timestamp = DateTime.UtcNow,
            Message = "Warm up"
        });

        transaction.Commit();
    }

    var sw = Stopwatch.StartNew();

    using (var session = sessionFactory.OpenStatelessSession())
    using (var transaction = session.BeginTransaction())
    {
        for (var i = 0; i < numberItems; ++i)
        {
            session.Insert(new LogEntry()
            {
                HostName = "host",
                UserName = "user",
                LogName = "log",
                Level = 0,
                Thread = "thread",
                Timestamp = DateTime.UtcNow,
                Message = "Message " + i
            });
        }

        transaction.Commit();
    }

    return sw.Elapsed;
}

public TimeSpan WriteWithBC(ISessionFactory sessionFactory, int numberItems, int batchSize)
{
    using (var session = sessionFactory.OpenStatelessSession())
    using (var bulkCopy = new SqlBulkCopy((SqlConnection)session.Connection))
    {
        bulkCopy.BatchSize = batchSize;
        bulkCopy.DestinationTableName = "LogEntries";
        var table = new DataTable("LogEntries");
        table.Columns.Add("Id", typeof(int));
        table.Columns.Add("HostName", typeof(string));
        table.Columns.Add("UserName", typeof(string));
        table.Columns.Add("LogName", typeof(string));
        table.Columns.Add("Timestamp", typeof(DateTime));
        table.Columns.Add("Level", typeof(int));
        table.Columns.Add("Thread", typeof(string));
        table.Columns.Add("Message", typeof(string));

        var row = table.NewRow();
        row["HostName"] = "host";
        row["UserName"] = "user";
        row["LogName"] = "log";
        row["Timestamp"] = DateTime.UtcNow;
        row["Level"] = 0L;
        row["Thread"] = "thread";
        row["Message"] = "Warm up";
        table.Rows.Add(row);

        bulkCopy.WriteToServer(table);
    }

    var sw = Stopwatch.StartNew();

    using (var session = sessionFactory.OpenStatelessSession())
    using (var bulkCopy = new SqlBulkCopy((SqlConnection)session.Connection))
    {
        bulkCopy.BatchSize = batchSize;
        bulkCopy.DestinationTableName = "LogEntries";
        var table = new DataTable("LogEntries");
        table.Columns.Add("Id", typeof(int));
        table.Columns.Add("HostName", typeof(string));
        table.Columns.Add("UserName", typeof(string));
        table.Columns.Add("LogName", typeof(string));
        table.Columns.Add("Timestamp", typeof(DateTime));
        table.Columns.Add("Level", typeof(int));
        table.Columns.Add("Thread", typeof(string));
        table.Columns.Add("Message", typeof(string));

        for (var i = 0; i < numberItems; ++i)
        {
            var row = table.NewRow();
            row["HostName"] = "host";
            row["UserName"] = "user";
            row["LogName"] = "log";
            row["Timestamp"] = DateTime.UtcNow;
            row["Level"] = 0;
            row["Thread"] = "thread";
            row["Message"] = "Message " + i;
            table.Rows.Add(row);
        }

        bulkCopy.WriteToServer(table);
    }

    return sw.Elapsed;
}

使用NHibernate执行插入时,以下是一些示例输出:

Saving 10000 items with batch size 500: 00:00:12.3064936
Saving 10000 items with batch size 100: 00:00:12.3600981
Saving 10000 items with batch size 1: 00:00:12.8102670

作为比较点,您将看到我还实施了基于BCP的解决方案。以下是一些示例输出:

Saving 10000 items with batch size 500: 00:00:00.3142613
Saving 10000 items with batch size 100: 00:00:00.6757417
Saving 10000 items with batch size 1: 00:00:26.2509605

显然,BCP解决方案比NH更快。同样明显的是,配料正在影响BCP溶液的速度,但不影响NH的解决方案。使用NHibernate进行插入时,NHProf显示以下内容:

alt text http://img9.imageshack.us/img9/8407/screenshotac.png

只有INSERT s,没有SELECT。有趣的是,NHProf在任何时候都不会给我this warning

我已经尝试在我的配置文件和代码中根据上面的测试用例指定adonet.batch_size

现在,我不期望NH解决方案能够达到BCP解决方案的速度,但我至少想知道为什么批处理不起作用。如果启用批处理已经足够好了,那么我可以在BCP上使用NH解决方案,以保持代码库更简单。

任何人都可以解释为什么NH拒绝遵守ADO.NET批处理,以及我可以做些什么来修复它?我读过的所有分散的NH“文档”都指出,您需要做的就是指定adonet.batch_size并且(最好)使用无状态会话,但我正在做这两件事。

由于

2 个答案:

答案 0 :(得分:12)

使用identity中断批处理。

Fabio Maulo解释here

最好的选择是切换到不同的生成器(我总是建议hiloguid.comb

答案 1 :(得分:3)

如果您使用identity作为主键,则ISessionIStatelessSession都无法批量插入。

当插入时,Nhibernate会将正确的值放入您的Id属性中。但是当您使用Identity时,唯一可以获取此ID的地方是数据库。使用hilo进行批量插入。