从二进制文件导入数据非常慢

时间:2014-10-16 20:24:32

标签: performance vba excel-vba binary excel

我有一个包含测试数据的文件夹.txt文件。我编写了一个宏来筛选.txt文件,根据一些搜索条件提取我想要的信息,然后将这些搜索结果写入二进制文件。

所以现在我有一个包含简化数据集的二进制文件,我写了另一个宏来搜索二进制文件,以寻找我真正想要的东西。

由于某种原因,我的宏读取二进制文件中的数据非常慢。

作为比较,我编写了一个宏来查看特定搜索的所有txt文件,将其写入二进制文件,然后将其读回Excel。那只用了60秒。

这是一个片段。我想知道if-else语句是否符合我的搜索条件[LC(a)和EID(e)]是否会降低速度或二进制文件的大小(仅200 MB)。

Type MyBinaryRecordInfo
   MyBinaryRecordInfo1(1 To 12) As String ' variable length
End Type    

i = 1
Open currentpath & "\" & bin_fname & ".DAT" For Binary As #f
' read records from the binary file
For a = 1 To totalLC
    For e = 1 To totalElm
        Do While Loc(f) < LOF(f)
            Call ReadBinRecord(MyRecord, f, ElmType)
            Sheets(ElmType).Select
            If MyRecord.MyBinaryRecordInfo1(1) = LC(a) Then
                If MyRecord.MyBinaryRecordInfo1(2) = EID(e) Then
                    For j = 1 To totalbinrec
                        With MyRecord
                            Cells(i + 3, j) = .MyBinaryRecordInfo1(j)
                        End With
                    Next j
                    i = i + 1
                    Exit Do
                End If
            End If
        Loop
    Next e
Next a
Close #f ' close the file

Sub ReadBinRecord(MyRecord As MyBinaryRecordInfo, f As Integer, ElmType As String)
' reads the next record from an open binary file
Dim intSize As Integer

For j = 1 To totalbinrec
   With MyRecord
       Get f, , intSize ' read the size of the ID field
       .MyBinaryRecordInfo1(j) = String(intSize, " ") ' set the variable length
       Get f, , .MyBinaryRecordInfo1(j) ' read the variable string field
   End With
Next j

1 个答案:

答案 0 :(得分:0)

可能缓慢的部分是写出Excel。特别是,每次读取行时都会更改工作表

假设您事先知道二进制文件中的行数,您可以读入内存,然后在结束时写出一次。代码看起来如下所示:

Option Explicit

Type MyBinaryRecordInfo
   MyBinaryRecordInfo1(1 To 12) As String ' variable length
End Type

Sub x()
num_rows = 5000
Open currentpath & "\" & bin_fname & ".DAT" For Binary As #f
Dim V(1 To totalElm) As Variant
Dim V2(1 To num_rows, 1 To 12) As Variant
For e = 1 To totalElm
  V(e) = V2
Next e
' read records from the binary file
For a = 1 To totalLC
    For e = 1 To totalElm
        Do While Loc(f) < LOF(f)
            Call ReadBinRecord(MyRecord, f, ElmType)
            If MyRecord.MyBinaryRecordInfo1(1) = LC(a) Then
                If MyRecord.MyBinaryRecordInfo1(2) = EID(e) Then
                    For j = 1 To totalbinrec
                        With MyRecord
                            V(e)(i, j) = .MyBinaryRecordInfo1(j)
                        End With
                    Next j
                    i = i + 1
                    Exit Do
                End If
            End If
        Loop
    Next e
Next a
Close #f ' close the file

' write out
For e = 1 To totalElm
  Sheets(ElmType).Select
  Cells(3, 1).Resize(num_rows, 12).Value = V(e)
Next e
End Sub

Sub ReadBinRecord(MyRecord As MyBinaryRecordInfo, f As Integer, ElmType As String)
' reads the next record from an open binary file
Dim intSize As Integer

For j = 1 To totalbinrec
   With MyRecord
       Get f, , intSize ' read the size of the ID field
       .MyBinaryRecordInfo1(j) = String(intSize, " ") ' set the variable length
       Get f, , .MyBinaryRecordInfo1(j) ' read the variable string field
   End With
Next j
End Sub

如果您不知道行数,那么您只需每隔500行重新保留内部变体一次。

由于这会在最后写出所有内容,您可能会发现使用Application.Statusbar =&#34; my string&#34;写一个进度信息