从CSV文件中提取数据到可搜索数据结构的最佳方法是什么?

时间:2010-06-25 09:59:31

标签: vb.net csv

我有一个包含48列数据的csv文件。 我需要打开此文件,将其放入数据结构中,然后搜索该数据并将其显示在DataRepeater中。

到目前为止,我已成功使用CSVReader提取数据并将其绑定到myDataRepeater。但是我现在正在努力将数据放在一个表中,以便我可以过滤结果。我不想使用SQL或任何其他数据库。

有没有人对最佳方法有什么建议?

到目前为止,这是在返回所有记录:

Private Sub BindCsv()
    ' open the file "data.csv" which is a CSV file with headers"
    Dim dirInfo As New DirectoryInfo(Server.MapPath("~/ftp/"))
    Dim fileLocation As String = dirInfo.ToString & "data.txt"
    Using csv As New CsvReader(New StreamReader(fileLocation), True)
        myDataRepeater.DataSource = csv
        myDataRepeater.DataBind()
    End Using
End Sub

Protected Sub myDataRepeater_ItemDataBound(ByVal sender As Object, ByVal e As RepeaterItemEventArgs) Handles myDataRepeater.ItemDataBound

    Dim dataItem As String() = DirectCast(e.Item.DataItem, String())

    DirectCast(e.Item.FindControl("lblPropertyName"), ITextControl).Text = dataItem(2).ToString
    DirectCast(e.Item.FindControl("lblPrice"), ITextControl).Text = dataItem(7).ToString

    DirectCast(e.Item.FindControl("lblPricePrefix"), ITextControl).Text = dataItem(6)
    DirectCast(e.Item.FindControl("lblPropertyID"), ITextControl).Text = dataItem(1)
    DirectCast(e.Item.FindControl("lblTYPE"), ITextControl).Text = dataItem(18)
    DirectCast(e.Item.FindControl("lblBedrooms"), ITextControl).Text = dataItem(8)
    DirectCast(e.Item.FindControl("lblShortDescription"), ITextControl).Text = dataItem(37)

    Dim dirInfo As New DirectoryInfo(Server.MapPath("~/ftp/images/"))
    DirectCast(e.Item.FindControl("imgMain"), Image).ImageUrl = dirInfo.ToString & "pBRANCH_" & dataItem(1) & ".jpg"
    DirectCast(e.Item.FindControl("linkMap"), HyperLink).NavigateUrl = "http://www.multimap.com/map/browse.cgi?client=public&db=pc&cidr_client=none&lang=&pc=" & dataItem(5) & "&advanced=&client=public&addr2=&quicksearch=" & dataItem(5) & "&addr3=&addr1="
End Sub

代码添加到过滤结果:

 Try
        Dim csv As New CSVFile(fileLocation)
        Dim ds As DataSet = csv.ToDataSet("MyTable")
        If Not ds Is Nothing Then


            Dim strExpr As String = "Bedrooms >= '3'"
            Dim strSort As String = "PropertyID ASC"

            'Use the Select method to find all rows matching the filter.
            Dim myRows() As DataRow
            'myRows = Dt.Select(strExpr, strSort)
            myRows = csv.ToDataSet("MyTable").Tables("MyTable").Select(strExpr, strSort)
            myDataRepeater.DataSource = myRows
            myDataRepeater.DataBind()
        End If
    Catch ex As Exception

    End Try

哪个返回我期待的两行但是当它绑定到datarepeater时我得到以下错误: DataBinding:'System.Data.DataRow'不包含名为'PropertyName'的属性。

更正后的代码,未应用过滤器:

Public Sub PageLoad(ByVal Sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
    If Not Page.IsPostBack Then
        ReadCsv()
        lblSearch.Text = "Lettings Search"
    End If
End Sub

Private Sub ReadCsv()
    Dim dirInfo As New DirectoryInfo(Server.MapPath("~/ftp/"))
    Dim fileLocation As String = dirInfo.ToString & "data.txt"

    Try
        Dim csv As New CSVFile(fileLocation)
        Dim ds As DataSet = csv.ToDataSet("MyTable")
        If Not ds Is Nothing Then
            myDataRepeater.DataSource = ds
            myDataRepeater.DataMember = ds.Tables.Item(0).TableName
            myDataRepeater.DataBind()
        End If
        ds = Nothing
        csv = Nothing
    Catch ex As Exception
        MsgBox(ex.Message)
    End Try
End Sub

Protected Sub btnSubmit_Click(ByVal sender As Object, ByVal e As System.Web.UI.ImageClickEventArgs) Handles btnSubmit.Click
    Dim rowCount As Integer

    rowCount = QueryCsv()
    pnlSearch.Visible = False
    lblResults.Visible = True
    lblSearch.Text = "Search Results"
    lblResults.Text = "Your search returned " & rowCount.ToString & " results"

    If rowCount > 0 Then
        myDataRepeater.Visible = True
        pnlResults.Visible = True
        btnBack.Visible = True
    End If

End Sub

Protected Function QueryCsv() As Integer

    Dim dirInfo As New DirectoryInfo(Server.MapPath("~/ftp/"))
    Dim fileLocation As String = dirInfo.ToString & "data.txt"
    Dim numberofRows As Integer

    Try
        Dim csv As New CSVFile(fileLocation)
        Dim ds As DataSet = csv.ToDataSet("MyTable")
        If Not ds Is Nothing Then


            Dim strExpr As String = "PropertyID = 'P1005'"
            Dim strSort As String = "PropertyID DESC"

            Try

                ds.Tables.Item(0).DefaultView.RowFilter = strExpr
                ds.Tables.Item(0).DefaultView.Sort = strSort
                myDataRepeater.DataSource = ds.Tables.Item(0).DefaultView

            Catch ex As Exception
            End Try

        End If
    numberofRows = ds.Tables("MyTable").Rows.Count
    Catch ex As Exception

    End Try
    Return numberofRows
End Function

2 个答案:

答案 0 :(得分:1)

为什么不使用内置的TextFileParser将数据导入DataTable?像Paul Clement在this thread

中的回答

答案 1 :(得分:0)

我做到这一点的方法之一是使用结构数组和反射。

首先,在模块中设置结构:CSVFileFields.vb


Imports System.Reflection

Public Module CSVFileFields
 #Region " CSV Fields "
    Public Structure CSVFileItem
        Dim NAME As String
        Dim ADDR1 As String
        Dim ADDR2 As String
        Dim CITY As String
        Dim ST As String
        Dim ZIP As String
        Dim PHONE As String

        Public Function FieldNames() As String()
            Dim rtn() As String = Nothing
            Dim flds() As FieldInfo = Me.GetType.GetFields(BindingFlags.Instance Or BindingFlags.Public)
            If Not flds Is Nothing Then
                ReDim rtn(flds.Length - 1)
                Dim idx As Integer = -1
                For Each fld As FieldInfo In flds
                    idx += 1
                    rtn(idx) = fld.Name
                Next
            End If
            Return rtn
        End Function

        Public Function ToStringArray() As String()
            Dim rtn() As String = Nothing
            Dim flds() As FieldInfo = Me.GetType.GetFields(BindingFlags.Instance Or BindingFlags.Public)
            If Not flds Is Nothing Then
                ReDim rtn(flds.Length - 1)
                Dim idx As Integer = -1
                For Each fld As FieldInfo In flds
                    idx += 1
                    rtn(idx) = fld.GetValue(Me)
                Next
            End If
            Return rtn
        End Function

        Public Shadows Function ToString(ByVal Delimiter As String) As String
            Dim rtn As String = ""
            Dim flds() As FieldInfo = Me.GetType.GetFields(BindingFlags.Instance Or BindingFlags.Public)
            If Not flds Is Nothing Then
                For Each fld As FieldInfo In flds
                    rtn &= fld.GetValue(Me) & Delimiter
                Next
                rtn = rtn.Substring(0, rtn.Length - 1)
            End If
            Return rtn
        End Function
    End Structure

#End Region 结束模块

接下来,我们将从刚刚制作的结构中创建自己的集合。这将使我们的结构易于使用.Add()。Remove()等。我们还可以使用.RemoveAt(索引)删除单个项目。文件:CSVFileItemCollection.vb


 #Region " CSVFileItem Collection "

Public Class CSVFileItemCollection Inherits System.Collections.CollectionBase

Public Sub Add(ByVal NewCSVFileItem As CSVFileItem)
    Me.List.Add(NewCSVFileItem)
End Sub

Public Sub Remove(ByVal RemoveCSVFileItem As CSVFileItem)
    Me.List.Remove(RemoveCSVFileItem)
End Sub

Default Public Property Item(ByVal index As Integer) As CSVFileItem
    Get
        Return Me.List.Item(index)
    End Get
    Set(ByVal value As CSVFileItem)
        Me.List.Item(index) = value
    End Set
End Property

Public Shadows Sub Clear()
    MyBase.Clear()
End Sub

Public Shadows Sub RemoveAt(ByVal index As Integer)
    Remove(Item(index))
End Sub

End Class

#End Region

接下来,您需要您的类来处理反射导入:CSVFile.vb

Public Sub Add(ByVal NewCSVFileItem As CSVFileItem)
    Me.List.Add(NewCSVFileItem)
End Sub

Public Sub Remove(ByVal RemoveCSVFileItem As CSVFileItem)
    Me.List.Remove(RemoveCSVFileItem)
End Sub

Default Public Property Item(ByVal index As Integer) As CSVFileItem
    Get
        Return Me.List.Item(index)
    End Get
    Set(ByVal value As CSVFileItem)
        Me.List.Item(index) = value
    End Set
End Property

Public Shadows Sub Clear()
    MyBase.Clear()
End Sub

Public Shadows Sub RemoveAt(ByVal index As Integer)
    Remove(Item(index))
End Sub

好的一点解释。这个类正在做的是首先获取分隔(“,”)文本行并将其拆分为字符串数组。然后,它遍历结构CSVFileItem中的每个字段,并根据索引填充该结构变量。你有多少物品并不重要。只要声明结构的顺序与加载的内容相同,就可以有1或1,000。例如,您的输入CSV应与CSVFileItem匹配为“Name,Address1,Address2,City,State,Zip,Phone”。这是通过上面的代码来完成的:


Imports System.Reflection
Imports System.IO
Imports Microsoft.VisualBasic.PowerPacks

Public Class CSVFile #Region " Private Variables " Private _CSVFile As CSVFileItem, _Delimiter As String, _Items As New CSVFileItemCollection #End Region

#Region " Private Methods " Private Sub FromString(ByVal Line As String, ByVal Delimiter As String) Dim CSVFileElements() As String = Line.Split(Delimiter) If Not CSVFileElements Is Nothing Then Dim fldInfo() As FieldInfo = _CSVFile.GetType.GetFields(BindingFlags.Instance Or BindingFlags.Public) If Not fldInfo Is Nothing Then Dim itm As System.ValueType = CType(_CSVFile, System.ValueType) For fldIdx As Integer = 0 To CSVFileElements.Length - 1 fldInfo(fldIdx).SetValue(itm, CSVFileElements(fldIdx).Replace(Chr(34), "")) Next _CSVFile = itm Else Dim itms As Integer = 0 If Not fldInfo Is Nothing Then itms = fldInfo.Length End If Throw New Exception("Invalid line definition.") End If Else Dim itms As Integer = 0 If Not CSVFileElements Is Nothing Then itms = CSVFileElements.Length End If Throw New Exception("Invalid line definition.") End If End Sub #End Region

#Region " Public Methods " Public Sub New() _CSVFile = New CSVFileItem End Sub

Public Sub New(ByVal Line As String, ByVal Delimiter As String)
    _CSVFile = New CSVFileItem
    _Delimiter = Delimiter
    FromString(Line, Delimiter)
End Sub

Public Sub New(ByVal Filename As String)
    LoadFile(Filename)
End Sub

Public Sub LoadFile(ByVal Filename As String)
    Dim inFile As StreamReader = File.OpenText(Filename)
    Do While inFile.Peek > 0
        FromString(inFile.ReadLine, ",")
        _Items.Add(_CSVFile)
        _CSVFile = Nothing
    Loop
    inFile.Close()
End Sub

#End Region

#Region " Public Functions " Public Function ToDataSet(ByVal TableName As String) As DataSet Dim dsCSV As DataSet = Nothing If Not _Items Is Nothing AndAlso _Items.Count > 0 Then Dim flds() As FieldInfo = _Items.Item(0).GetType.GetFields(BindingFlags.Instance Or BindingFlags.Public) If Not flds Is Nothing Then dsCSV = New DataSet dsCSV.Tables.Add(TableName) For Each fld As FieldInfo In flds 'Add Column Names With dsCSV.Tables.Item(TableName) .Columns.Add(fld.Name, fld.FieldType) End With Next

            'Populate Table with Data
            For Each itm As CSVFileItem In _Items
                dsCSV.Tables.Item(TableName).Rows.Add(itm.ToStringArray)
            Next
        End If
    End If
    Return dsCSV
End Function

#End Region

#Region " Public Properties " Public ReadOnly Property Item() As CSVFileItem Get Return _CSVFile End Get End Property

Public ReadOnly Property Items() As CSVFileItemCollection
    Get
        Return _Items
    End Get
End Property

#End Region End Class

为了简化操作,我们可以简单地将文件传递给文件路径,而不必从主类加载文件,这个类将完成所有工作并返回我们结构的集合。我知道这看起来像很多设置,但它是值得的,你可以回来并将你的原始结构改为任何东西,其余的代码仍然可以完美地工作!

现在让一切顺利。现在,您只需几行代码即可了解这是多么容易实现。文件:frmMain.vb

Public Sub New(ByVal Line As String, ByVal Delimiter As String)
    _CSVFile = New CSVFileItem
    _Delimiter = Delimiter
    FromString(Line, Delimiter)
End Sub

Public Sub New(ByVal Filename As String)
    LoadFile(Filename)
End Sub

Public Sub LoadFile(ByVal Filename As String)
    Dim inFile As StreamReader = File.OpenText(Filename)
    Do While inFile.Peek > 0
        FromString(inFile.ReadLine, ",")
        _Items.Add(_CSVFile)
        _CSVFile = Nothing
    Loop
    inFile.Close()
End Sub

这确实可以实现一些动态编程。您可以将它们包装在泛型类中,并为任何结构调用函数。然后,您将获得一些可重复使用的代码,这些代码可以使您的程序高效并减少编程时间!

编辑:

添加了将结构集合转储到数据集,然后动态填充datarepeater的功能。

希望这会有所帮助。 (我知道这是很多信息,看起来很多工作,但我向你保证,一旦你实现这一点,它将真正减少未来的项目编码时间!)