我有两个列表(每个列表在80000到200000行之间)。 List1:列出这样的OLD数据:sku | price | stock 清单2:列出这样的新数据:sku | price | stock
我现在从list1中选择第1行,将该行拆分为3个文本框(sku,价格和库存)。接下来,我使用textbox1并在list2中搜索该sku。当我发现这条线时,我把它分成文本框。接下来我比较价格和库存,做一些计算(额外的东西与更多的文本框我不会提到,以保持这篇文章干净)。如果价格或库存没有变化,我将从list1的第1行创建的文本框放在一起,并将该行放在新文件List3中。如果有变化,我使用从list2创建的行中的文本框,并将该行放入List3文件。 所有人都在努力,但速度不够快。我已经对list1和list2中的行进行了排序,所以我不必循环多行来找到我需要的行。在每次编辑/比较之后,我删除list1和list2中使用的行以使列表更小并且通过这样做也更快。但它仍然不够快。 什么不够快?我在list1和list2中用25000行进行测试。我已达到每小时超过14000的速度。但是当我增加超过100000行的列表时,我每小时只能达到4000。每小时14000不够快,但每小时4000更快。 (我不是专家,但我认为我可以使用visual basic来获得更好的速度)
这里有一些我用来做魔术的代码(比如我如何读写文件):
计时器:我认为人们会打击我使用它们,但这就是它现在如何运作;-) 定时器1将用于从list1抓取第一行并将其拆分。这不是耗时的,所以不是问题,但这里有一些来自timer1的代码:
Dim fileReader As System.IO.StreamReader
fileReader =
My.Computer.FileSystem.OpenTextFileReader(Path.GetDirectoryName(Application.ExecutablePath) & "\file-with-old-prices-stock.txt")
Dim stringReader As String
stringReader = fileReader.ReadLine()
TextBox1.Text = stringReader
Dim myline As String
Dim half() As String
myline = TextBox1.Text
half = myline.Split("|")
TextBox1.Text = half(0)
TextBox16.Text = half(1)
TextBox17.Text = half(2)
接下来我启动timer2: 这是完整的timer2代码:
Timer2.Stop()
Dim appPath As String = Path.GetDirectoryName(Application.ExecutablePath)
Dim fileReader As New System.IO.StreamReader(appPath & "\file-with-new-prices-stock.txt")
Dim fileContents() As String = fileReader.ReadToEnd.Split(ControlChars.Cr)
TextBox3.Text = fileContents.Length
Dim i As Integer
Do Until i = "1"
If TextBox7.Text = TextBox3.Text Then
NumericUpDown1.Value = 1
Timer2.Stop()
i = "1"
Else
If fileContents(TextBox7.Text).Contains(TextBox1.Text) Then
TextBox2.Text = fileContents(TextBox7.Text)
Dim myline As String
Dim half() As String
myline = TextBox2.Text
half = myline.Split("|")
TextBox4.Text = half(0)
TextBox5.Text = half(1)
TextBox6.Text = half(2)
If TextBox6.Text > "1" Then
TextBox6.Text = "1"
End If
CheckBox1.CheckState = CheckState.Checked
fileReader.Close()
Do Until i = "1"
Dim fs As System.IO.FileStream
Dim fileInUse As Boolean = True
Try
'if Open() succeeds, then we know the file is not currently in use
fs = System.IO.File.Open(appPath & "\file-with-new-prices-stock.txt", FileMode.Open, FileAccess.Read, FileShare.None)
fileInUse = False
fs.Close()
Dim linesList As New List(Of String)(File.ReadAllLines(Path.GetDirectoryName(Application.ExecutablePath) & "\file-with-new-prices-stock.txt"))
'Remove the line to delete, e.g.
linesList.RemoveAt(TextBox7.Text)
File.WriteAllLines(Path.GetDirectoryName(Application.ExecutablePath) & "\file-with-new-prices-stockTEMP.txt", linesList.ToArray())
My.Computer.FileSystem.CopyFile(Path.GetDirectoryName(Application.ExecutablePath) & "\file-with-new-prices-stockTEMP.txt", Path.GetDirectoryName(Application.ExecutablePath) & "\file-with-new-prices-stock.txt", True)
fileReader.Close()
Timer200.Start()
Timer2.Stop()
i = "1"
Catch ex As Exception
End Try
Loop
End If
If Not fileContents(TextBox7.Text).Contains(TextBox1.Text) Then
TextBox7.Text = TextBox7.Text + 1
fileReader.Close()
Timer2.Start()
End If
End If
Loop
所以我所做的就是使用textbox1.text并以某种方式在list2中找到该文本的行(file-with-new-prices-stock.txt)。这是缓慢(或没有正确编码的部分。我假设的部分)。
接下来,我使用2行(我使用几个计时器)做魔术并创建一个新行并将该行附加到list3(工作正常)。
当我无法从list2中的list1找到sku时,它会变慢。应用程序需要逐行读取所有行。我试图将计时器加倍并拆分文件,但这样我只是拆分文件,应用程序将像以前一样慢/快。
我找到但无法让他们工作的解决方案是背景工作者和多跋涉。当我使用这两个选项中的一个时,我使用文本框并获取错误。如果不需要,我不想完全重写我的应用程序。也许有一种更快的方法来预读list2文件或者在搜索sku时没有读取完整的行(例如当sku以“1”开头时它会跳过以其他东西开头的行。)
喜欢看到一些建议,告诉我需要走的路;-) 谢谢!
答案 0 :(得分:0)
您可以采取一些措施来提高代码的速度和可读性。
我认为在这个简单的示例代码中可能会添加新的SKU:
Option Infer On
Imports System.IO
Imports Microsoft.VisualBasic.FileIO
Module Module1
Private Class ItemData
Property Price As Decimal
Property StockLevel As Integer
End Class
Private Sub GenerateTestData(filename As String, quantity As Integer, price As Decimal)
' Generate some test data. It is not at all sophisticated.
Dim rand As New Random
Using sw As New StreamWriter(filename)
Dim skuNumber As Integer = quantity
Dim sku As String
For i = 1 To quantity
sku = "A" & skuNumber.ToString()
sw.WriteLine(String.Format("{0}|{1}|{2}", sku, price, rand.Next(0, 101)))
skuNumber -= 1
Next
End Using
End Sub
Private Function ReadData(filename As String) As Dictionary(Of String, ItemData)
Dim data As New Dictionary(Of String, ItemData)
Using tfp As New TextFieldParser(filename)
tfp.SetDelimiters({"|"})
While Not tfp.EndOfData
Dim parts = tfp.ReadFields()
' Make sure to convert the text input to the correct types.
' TODO: check there are three parts.
' TODO: make sure the price and stocklevel fields can be parsed.
data.Add(parts(0), New ItemData With {.Price = CDec(parts(1)), .StockLevel = CInt(parts(2))})
End While
End Using
Return data
End Function
Private Sub UpdateData(currentDataFile As String, updateDataFile As String, newDataFile As String)
Dim currentData = ReadData(currentDataFile)
Dim updateData = ReadData(updateDataFile)
' Update or add data.
For Each item In updateData
If currentData.ContainsKey(item.Key) Then
currentData(item.Key) = item.Value
Else
currentData.Add(item.Key, item.Value)
End If
Next
' Write updated info to the new file.
Using sw As New StreamWriter(newDataFile)
For Each item In currentData
sw.WriteLine(String.Format("{0}|{1}|{2}", item.Key, item.Value.Price, item.Value.StockLevel))
Next
End Using
End Sub
Sub Main()
Dim dataFolder = "C:\temp"
Dim currentFile = Path.Combine(dataFolder, "currentData.txt")
Dim updateFile = Path.Combine(dataFolder, "updateData.txt")
GenerateTestData(currentFile, 100000, 5.99D)
GenerateTestData(updateFile, 90000, 4D)
Dim newFile = Path.Combine(dataFolder, "newData.txt")
' TODO: remove timing parts.
Dim sw As New Stopwatch
sw.Start()
UpdateData(currentFile, updateFile, newFile)
sw.Stop()
Console.WriteLine((sw.ElapsedMilliseconds / 1000).ToString())
Console.ReadLine()
End Sub
End Module
我的电脑在一分钟内就完成了。