使用ADO将大型csv文件导入mdb时出现性能问题

时间:2014-01-20 17:05:24

标签: vba excel-vba csv fastercsv excel

这是我的代码,但需要大约一个小时才能将所有1700万行导出到mdb中。我不能为此目的使用mysql或sql server。我必须在访问数据库中执行此操作并快速执行此过程在一周内运行一次。 Plz建议可用于此任务的最快方法

Sub insertDataIntoMDB()
 Dim Dbfilepath As String
 Set cnn = CreateObject("ADODB.Connection")
 Set rs = CreateObject("ADODB.Recordset")
 Set rst = CreateObject("ADODB.Recordset")

        Dim arrData() As String
        Dim s As String
        Dim i As Integer


        Dbfilepath = ThisWorkbook.Path & "\DB\Interface.accdb"
        cnn.Open "Provider= Microsoft.ACE.OLEDB.12.0;" & " Data Source=" & Dbfilepath & ";" & "Persist Security Info =False;"

        q1 = "DELETE * FROM MYTABLE"
        Set rs = cnn.Execute(q1)

        'q1 = "ALTER TABLE MyTable ALTER COLUMN ID autonumber(1,1)"
        'Set rs = cnn.Execute(q1)

        p = UserForm1.csvFolder & "\" & sItem & ".csv"

        Open p For Input As #1
        Do While Not EOF(1)

          Close #1


            Line Input #1, s
            arrData = Split(s, ",")
           q1 = "INSERT INTO MyTable(F1,F2,F3) values(" & arrData(0) & "," & arrData(1) & "," & arrData(2) & ")"
           Set rst = cnn.Execute(q1)
        Loop
        Close #1
        rs.Close
        rs`enter code here`t.Close
        Set rst = Nothing
        cnn.Close
        Set rs = Nothing
        Set cnn = Nothing


End Sub

1 个答案:

答案 0 :(得分:0)

就在这里,你有一个巨大的减速和数据损坏的可能性。

q1 = "INSERT INTO MyTable(F1,F2,F3) values(" & arrData(0) & "," & arrData(1) & "," & arrData(2) & ")"
Set rst = cnn.Execute(q1)

字符串连接很慢,尤其是在VBA中。因此,只要您编写"something" & "something"并将其置于循环中,就会要求性能降低。

此外,DAO通常比使用ADO更快。

Read this answer
And maybe this question and its answers

如果您坚持使用ADO,则可能需要使用SELECT语句打开记录集,然后将数据附加到该记录集,然后调用UpdateBatch。

You can read more discussion here

祝你好运!