创建多个.txt文件,同时限制每个文件的大小

时间:2017-10-22 19:57:29

标签: vb.net io streamwriter

在我的计划中,我收集了大量信息,每个数十万到数百万行。我试图将我创建的每个文件限制到一定的大小,以便能够快速打开它并读取数据。我正在使用HashSet收集所有数据而不重复。

到目前为止,这是我的代码:

Dim Founds As HashSet(Of String)
Dim filename As String = (Environment.GetFolderPath(Environment.SpecialFolder.Desktop) + "\Sorted_byKING\sorted" + Label4.Text + ".txt")
Using writer As New System.IO.StreamWriter(filename)
    For Each line As String In Founds
        writer.WriteLine(line)
    Next
    Label4.Text = Label4.Text + 1   'Increments sorted1.txt, sorted2.txt etc
End Using

所以,我的问题是:

我如何进行保存,假设在文本文件中有250,000行,然后转移到另一个并添加下一个250,000?

1 个答案:

答案 0 :(得分:0)

首先,不要使用标签来简单地存储值。您应该使用变量,而不是变量的用途。

另一个建议是,总是使用Path.Combine来连接路径,这样你就不必担心路径的每个部分是否以分隔符结尾。

现在,回答你的问题:

如果您想逐行插入文字,可以使用以下内容:

Sub SplitAndWriteLineByLine()
    Dim Founds As HashSet(Of String) 'Don't forget to initialize and fill your HashSet
    Dim maxLinesPerFile As Integer = 250000

    Dim fileNum As Integer = 0
    Dim counter As Integer = 0
    Dim filename As String = String.Empty

    Dim writer As IO.StreamWriter = Nothing

    For Each line As String In Founds
        If counter Mod maxLinesPerFile = 0 Then
            fileNum += 1
            filename = IO.Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Desktop),
                                       $"Sorted_byKING\sorted{fileNum.ToString}.txt")
            If writer IsNot Nothing Then writer.Close()
            writer = New IO.StreamWriter(filename)
        End If
        writer.WriteLine(line)
        counter += 1
    Next
    writer.Dispose()
End Sub

但是,如果您要按原样插入HashSet 中的文本,您可能不需要逐行编写,而是可以编写每个"束&# 34;一次排队。您可以使用以下内容:

Sub SplitAndWriteAll()
    Dim Founds As HashSet(Of String) 'Don't forget to initialize and fill your HashSet
    Dim maxLinesPerFile As Integer = 250000

    Dim fileNum As Integer = 0
    Dim filename As String = String.Empty
    For i = 0 To Founds.Count - 1 Step maxLinesPerFile
        fileNum += 1
        filename = IO.Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Desktop),
                                   $"Sorted_byKING\sorted{fileNum.ToString}.txt")
        IO.File.WriteAllLines(filename, Founds.Skip(i).Take(maxLinesPerFile))
    Next
End Sub