VBA for excel;编辑非常大的文件

时间:2018-05-22 10:42:39

标签: excel-vba large-files large-data-volumes vba excel

我有一大堆带有AIS(送货)数据的日志文件。由于这些日志文件大约每天200Mb,我正在尝试将它们缩小以进行存档。文件如下所示:

244630075;under way ;128°'; 0.0kt;52.395290N;4.886883E;342.0°;511°;55s; 170418 000000;serial#1(A)[1]
244670835;under way ;128°'; 0.0kt;52.410140N;4.833700E;283.8°;511°;54s; 170418 000000;serial#1(B)[3]
244750830;under way ;128°'; 0.0kt;52.404563N;4.864063E;  0.0°;511°;55s; 170418 000000;serial#1(B)[1]
244900124;under way ;000°'; 7.1kt;52.426495N;4.780100E;279.4°;281°;56s; 170418 000000;serial#1(B)[2]
244670779;under way ;000°'; 0.0kt;52.420773N;4.801418E;330.9°;325°;58s; 170418 000000;serial#1(A)[1]
244660512;under way ;128°'; 0.0kt;52.402092N;4.781258E;268.3°;511°;54s; 170418 000000;serial#1(B)[1]
236202000;under way ;000°';11.7kt;52.477408N;4.462048E;285.4°;296°;55s; 170418 000000;serial#1(B)[1]
244690403;under way ;128°'; 0.0kt;52.400760N;4.891647E;  0.0°;511°;55s; 170418 000000;serial#1(A)[1]

每个文件大约有200万行。为了减小这些文件的大小,我想删除包含“0.0kt”的每一行,因为它代表了对我没用的信息。为此,我在Excel中编写了一个VBA脚本。我似乎有脚本为主要部分工作。它遍历文件并编辑包含“0.0kt”的所有行。但是当脚本结束时,应该保存它导出一个空文件。

这是我的剧本:

Sub test()
'this will force the script to end when end of file is reached
On Error GoTo ASD

Const ForReading = 1
Const ForWriting = 2

Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objFile = objFSO.OpenTextFile("C:\x\170418.log", ForReading)

x = 1

Do
Do While i < 1000

        strline = objFile.ReadLine
         If InStr(strline, " 0.0kt") = 28 Then
            strline = "" & vbCrLf

        End If
    i = i + 1

Loop

'doevents and a calculation to call doevents after 1000 lines to prevent freezing of the script
DoEvents
a = a + 1
b = a * 1000
i = i + b
x = i / 1000
i = 0
iLineNumber = x

Loop

ASD:

objFile.Close

Set objFile = objFSO.OpenTextFile("C:\x\170418.log", ForWriting)
objFile.Write strline

objFile.Close

End Sub

保存并关闭包含“0.0kt”的所有行,而不是删除所有行的文件,我缺少什么?

由于

2 个答案:

答案 0 :(得分:1)

查看示例文本,我认为可以排除包含; 0.0kt;的任何行。

使用我已构建的内容,我已将其调整为拾取您的文件并每1000行使用DoEvents

Sub Test()

    Dim ifileno As Integer, ofileno As Integer, rownum As Long
    Dim ifilename As String, ofilename As String, excludestring As String, strLine As String

    ifilename = "C:\Users\v.doynov\Desktop\nd.txt"
    ofilename = "C:\Users\v.doynov\Desktop\nd_output.txt"
    excludestring = "; 0.0kt;"

    ifileno = FreeFile
    Open ifilename For Input As ifileno

    ofileno = FreeFile
    Open ofilename For Output As ofileno

    rownum = 0

    Do Until EOF(ifileno)
        rownum = rownum + 1
        Line Input #ifileno, strLine
        If InStr(strLine, excludestring) = 0 Then Print #ofileno, strLine
        If rownum Mod 1000 = 0 Then DoEvents
    Loop

    Close ifileno
    Close ofileno

End Sub

答案 1 :(得分:0)

随着代码的使用,我得出了类似的结论:

Sub TestMe()

    On Error GoTo ASD
    Dim objFSO As Object
    Dim objFile As Object
    Dim x&, i&, strLine$, a&, b&, iLineNumber&
    Const ForReading = 1
    Const ForWriting = 2

    Set objFSO = CreateObject("Scripting.FileSystemObject")
    Set objFile = objFSO.OpenTextFile("C:\Users\v.doynov\Desktop\nd.txt")

    x = 1
    Dim newString As String
    Do
        Do While i < 1000
            strLine = objFile.ReadLine
            If InStr(strLine, " 0.0kt") <> 29 Then 'Sample was 29 on my machine, not 28.
                newString = newString & strLine & vbCrLf
            End If
            i = i + 1
        Loop
    Loop

ASD:

    objFile.Close
    Set objFile = objFSO.OpenTextFile("C:\Users\v.doynov\Desktop\nd.txt", ForWriting)
    objFile.Write newString
    objFile.Close

End Sub

检查If InStr(strLine, " 0.0kt") <> 29 Then是否如此,它是否将行重新加到newString。最后,保存newString