在mdb数据库中更新超过50.000行的最快方法c#

时间:2014-07-03 07:53:58

标签: c# sql ms-access sql-update oledb

我在网上搜索了一些东西,但没有什么真正帮助我。我想用文章列表更新数据库,但我发现的方式非常慢。

这是我的代码:

List<Article> costs = GetIdCosts(); //here there are 70.000 articles
conn = new OleDbConnection(string.Format(MDB_CONNECTION_STRING, PATH, PSW));
conn.Open();
transaction = conn.BeginTransaction();

using (var cmd = conn.CreateCommand())
{
    cmd.Transaction = transaction;

    cmd.CommandText = "UPDATE TABLE_RO SET TABLE_RO.COST = ? WHERE TABLE_RO.ID = ?;";

    for (int i = 0; i < costs.Count; i++)
    {
        double cost = costs[i].Cost;
        int id = costs[i].Id;

        cmd.Parameters.AddWithValue("data", cost);
        cmd.Parameters.AddWithValue("id", id);

        if (cmd.ExecuteNonQuery() != 1) throw new Exception();
    }
}

transaction.Commit();

但这种方式需要花费很多时间,比如10分钟或更长时间。还有另一种加速更新的方法吗?感谢。

3 个答案:

答案 0 :(得分:2)

尝试将代码修改为:

List<Article> costs = GetIdCosts(); //here there are 70.000 articles

// Setup and open the database connection
conn = new OleDbConnection(string.Format(MDB_CONNECTION_STRING, PATH, PSW));
conn.Open();

// Setup a command
OleDbCommand cmd = new OleDbCommand();
cmd.Connection = conn;
cmd.CommandText = "UPDATE TABLE_RO SET TABLE_RO.COST = ? WHERE TABLE_RO.ID = ?;";

// Setup the paramaters and prepare the command to be executed
cmd.Parameters.Add("?", OleDbType.Currency, 255);
cmd.Parameters.Add("?", OleDbType.Integer, 8); // Assuming you ID is never longer than 8 digits

cmd.Prepare();
OleDbTransaction transaction = conn.BeginTransaction();
cmd.Transaction = transaction;

// Start the loop    
for (int i = 0; i < costs.Count; i++)
{
    cmd.Parameters[0].Value = costs[i].Cost;
    cmd.Parameters[1].Value = costs[i].Id;

    try
    {
        cmd.ExecuteNonQuery();
    }
    catch (Exception ex)
    {
        // handle any exception here
    }
}

transaction.Commit();
conn.Close();

cmd.Prepare方法会加快速度,因为它会在数据源上创建命令的编译版本。

答案 1 :(得分:1)

小变更选项:

使用StringBuilder和string.Format构造一个大命令文本。

var sb = new StringBuilder();

for(....){
  sb.AppendLine(string.Format("UPDATE TABLE_RO SET TABLE_RO.COST = '{0}' WHERE TABLE_RO.ID = '{1}';",cost, id));
}

更快的选择:

在第一个示例中构造一个sql,但这次使它看起来像(在结果中):

-- declaring table variable
declare table @data (id int primary key, cost decimal(10,8))  

-- insert union selected variables into the table
insert into @data
      select 1121 as id, 10.23 as cost
union select 1122 as id, 58.43 as cost
union select ...


-- update TABLE_RO using update join syntax where inner join data
-- and copy value from column in @data to column in TABLE_RO
update dest 
set dest.cost = source.cost
from TABLE_RO dest
inner join @data source on dest.id = source.id

这是您在不使用批量插入的情况下获得的最快速度。

答案 2 :(得分:0)

使用Ado.net和OleDb执行批量更新非常缓慢。如果可能,您可以考虑通过DAO执行更新。只需添加对DAO-Library(COM-Object)的引用,并使用类似下面的代码(注意 - &gt; untested):

// Import Reference to "Microsoft DAO 3.6 Object Library" (COM)

string TargetDBPath = "insert Path to .mdb file here";

DAO.DBEngine dbEngine = new DAO.DBEngine();
DAO.Database daodb = dbEngine.OpenDatabase(TargetDBPath, false, false, "MS Access;pwd="+"insert your db password here (if you have any)");

DAO.Recordset rs = daodb.OpenRecordset("insert target Table name here", DAO.RecordsetTypeEnum.dbOpenDynaset);

if (rs.RecordCount > 0)
{
    rs.MoveFirst();

    while (!rs.EOF)
    {
        // Load id of row
        int rowid = rs.Fields["Id"].Value;

        // Iterate List to find entry with matching ID
        for (int i = 0; i < costs.Count; i++)
        {
            double cost = costs[i].Cost;
            int id = costs[i].Id;

            if (rowid == id)
            {
                // Save changed values
                rs.Edit();
                rs.Fields["Id"].Value = cost;
                rs.Update();
            }
        }

        rs.MoveNext();
    }
}
rs.Close();

请注意我们在这里进行全表扫描。但是,除非表中的记录总数比更新记录的数量大许多个数量级,否则它仍应显着优于Ado.net方法...