Question

所以，我需要帮助找出如何使这项工作，并希望与其他VB专家开始对话。

我有一个每小时转储到导入文件夹的CSV文件。我编写了一个VB应用程序来扫描该文件夹，获取CSV文件并将其导入SQL数据库。没问题。

尝试在SQL中查询数据时出现问题。我需要的所有重要数据都在一个字段中，这是字段内容的修改示例：

Successful Write by DOMAIN\USERLOGIN on /SOMENETWORKSHARE/FOLDER/MyFolder/USERLOGIN/DesktopBackUp/log.txt

所以基本上我需要一种方法将上面的行拆分成多个字段，它需要在SQL导入时发生。

Successful Write“是事件”字段1

by“永远不会改变可以用作分隔符”

DOMAIN/USERLOGIN“DOMAIN不会更改，但userlogin会更改”字段2

on“永远不会改变可以用作分隔符”

/SOMENETWORKSHARE/FOLDER/MyFolder/USERLOGIN/DesktopBackUp/“这是文件路径”Field 3

log.txt“已修改的文件名”字段4

有时候没有文件名，只有文件路径。

我愿意接受建议，但CSV文件没有变化，我无法更改文件传送给我的方式。

先谢谢。

这是数据库结构

SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
SET ANSI_PADDING ON
GO
CREATE TABLE [dbo].[SACL](
    [Unique ID] [varchar](max) NULL,
    [Event ID] [varchar](max) NULL,
    [Event Time] [datetime] NULL,
    [Severity] [varchar](max) NULL,
    [Workspace] [varchar](max) NULL,
    [Headline] [varchar](max) NULL,
    [ID] [int] IDENTITY(1,1) NOT NULL,
    [Event] [varchar](max) NULL,
    [Username] [varchar](max) NULL,
    [Path] [varchar](max) NULL,
    [Filename] [varchar](max) NULL,
PRIMARY KEY CLUSTERED 
(
    [ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO
SET ANSI_PADDING OFF
GO

Answer 1

我希望这可以帮到你。调用此函数后，您可以使用结果将数据库中的每个字段包含在另一个表中或类似的东西中。

Public Function divideThis(CompleteString As String)

    Dim n As Integer = InStr(CompleteString, " by ")
    Dim Field1 As String = CompleteString.Substring(0, n - 1)

    Dim m As Integer = InStr(CompleteString, " on ")
    Dim Field2 As String = CompleteString.Substring(n + 3, m - n - 4)

    Dim o As Integer = InStrRev(CompleteString, "/")
    Dim Field3 As String = CompleteString.Substring(m + 3, o - m - 3)

    Dim Field4 As String = CompleteString.Substring(o, CompleteString.Length - o)

    Dim fragmented As String() = {Field1, Field2, Field3, Field4}

    Return (fragmented)

End Function

Answer 2

好的，试试这个...它将导入CSV文件，拆分所需的字段并合并它。

我已修改代码以在LINQ语句中包含RegEx来拆分该字段，如果没有文件名则它将起作用但是在结尾处是正斜杠并且这必须是唯一的字段（在你的CSV文件）包含正斜杠...原因是如果正斜杠丢失并且没有文件名你将丢失一个字段如果你的csv文件中的其他字段包含一个转发你会得到太多的字段斜线.....

Sub Main()  
    Dim regex As Text.RegularExpressions.Regex = New Text.RegularExpressions.Regex("\d+")  
    Dim readText() As String = File.ReadAllLines("<Path to your CSV file>")  

    Using connection As SqlConnection = New SqlConnection("<DB connection string>")  

        Dim command As SqlCommand = connection.CreateCommand()
        connection.Open()  

        command.CommandType = System.Data.CommandType.StoredProcedure  
        command.CommandText = "SP_InsertCSVData"  

        For i As Integer = 0 To readText.Count
            Dim readRecord() As String = readText.ToArray

            Dim CSVData As XElement = New XElement("Root",
                            From str In readText
                            Let fields = Text.RegularExpressions.Regex.Replace(str, "(by|on|\/(?=[^\/]*$))", ",").Split(",")
                            Select New XElement("CSVDataRecord",
                                New XAttribute("Field_1", fields(0)),
                                New XAttribute("Field_2", fields(1)),
                                New XAttribute("Field_3", fields(2)),
                                New XAttribute("Field_4", fields(3)),
                                New XAttribute("Field_5", fields(4)),
                                New XAttribute("Field_6", fields(5)),
                                New XAttribute("Field_7", fields(6)),
                                New XAttribute("Field_8", fields(7))
                            )
                        )

            command.Parameters.Clear()
            command.Parameters.Add(New SqlParameter With
            {
                .ParameterName = "@InputXML",
                .DbType = DbType.Xml,
                .Value = CSVData.CreateReader
            })

            command.ExecuteNonQuery()

        Next
    End Using

End Sub

这是新表...

/****** Object:  Table [dbo].[CSV_Import_Table]    Script Date: 06/02/2016 01:55:32 ******/   
SET ANSI_NULLS ON  
GO  

SET QUOTED_IDENTIFIER ON  
GO  

CREATE TABLE [dbo].[CSV_Import_Table](  
[Field_1] [nvarchar](max) NULL,  
[Field_2] [nvarchar](max) NULL,  
[Field_3] [nvarchar](max) NULL,  
[Field_4] [nvarchar](max) NULL,  
[Field_5] [nvarchar](max) NULL,  
[Field_6] [nvarchar](max) NULL,  
[Field_7] [nvarchar](max) NULL,  
[Field_8] [nvarchar](max) NULL  
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]  

GO

这是SP ......

/****** Object:  StoredProcedure [dbo].[SP_InsertCSVData]    Script Date: 06/02/2016 01:56:20 ******/  
SET ANSI_NULLS ON  
GO  
SET QUOTED_IDENTIFIER ON  
GO  
CREATE PROCEDURE [dbo].[SP_InsertCSVData] (@InputXML xml)  
as  
begin  
MERGE  CSV_Import_Table AS main  
USING (select Row.id.value('@Field_1','[nvarchar](MAX)') as Field_1, Row.id.value('@Field_2','[nvarchar](MAX)') as Field_2, Row.id.value('@Field_3','[nvarchar](MAX)') as Field_3, Row.id.value('@Field_4','[nvarchar](MAX)') as Field_4,  Row.id.value('@Field_5','[nvarchar](MAX)') as Field_5, Row.id.value('@Field_6','[nvarchar](MAX)') as Field_6, Row.id.value('@Field_7','[nvarchar](MAX)') as Field_7, Row.id.value('@Field_8','[nvarchar](MAX)') as Field_8  
            from @InputXML.nodes('/Root/CSVDataRecord') as Row(id)) as stage  
ON main.Field_1=stage.Field_1  
WHEN MATCHED THEN  
    UPDATE SET main.Field_1=stage.Field_1, main.Field_2=stage.Field_2, main.Field_3=stage.Field_3, main.Field_4=stage.Field_4 ,   main.Field_5=stage.Field_5 , main.Field_6=stage.Field_6, main.Field_7=stage.Field_7, main.Field_8=stage.Field_8     
WHEN NOT MATCHED THEN   
    INSERT  (Field_1, Field_2, Field_3, Field_4, Field_5, Field_6, Field_7, Field_8) VALUES (stage.Field_1, stage.Field_2, stage.Field_3, stage.Field_4, stage.Field_5, stage.Field_6, stage.Field_7, stage.Field_8);  
end

我猜你的CSV文件看起来像这样....

记录1字段1，DOMAIN \ USERLOGIN在/SOMENETWORKSHARE/FOLDER/MyFolder/USERLOGIN/DesktopBackUp/Log.txt成功写入，字段6，字段7，字段8

Answer 3

这是一个你应该能够在你的桌子上跑来测试的选择。

将FROM ( .... ) as tbl位替换为真实表

你可以看到选择中的表达式非常丑陋且难以处理，所以要考虑一下你是否乐意保持这个

最后有一个WHERE来阻止无效数据进入。如果有缺少分隔符的数据进入，则索引将最终为负数，并且您将收到各种错误。

对于这些事情，数据质量始终是一个问题。因此，我们不希望通过假设域名将永远具有相同的名称来复合，或者假设您只是从一个域中导入数据。

所以你需要在你的桌子上运行它，看它是否有效。如果是这样，下一步是：

从路径中取出文件名
更改为更新

这应该是直截了当的

SELECT 
SRC,
LEFT(SRC,CHARINDEX(' by ',SRC)) AS Operation,
SUBSTRING(
    SRC,
    CHARINDEX(' by ',SRC)+4,
    CHARINDEX(' on ',SRC) - CHARINDEX(' by ',SRC)-4
    ) AS LoginName,

RIGHT(
    SRC,  
    LEN(SRC) - CHARINDEX(' on ',SRC)-4
) AS FullPath,

REVERSE(LEFT(
    REVERSE(SRC),
    CHARINDEX('/',REVERSE(SRC))-1)
    ) AS PathAndFileName
FROM 
(
SELECT 'Successful Write by DOMAIN\USERLOGIN on /SOMENETWORKSHARE/FOLDER/MyFolder/USERLOGIN/DesktopBackUp/log.txt' AS SRC
UNION ALL
SELECT 'some invalid stuff' AS SRC
) as tbl
WHERE SRC LIKE '% by %\% on %/%'

应用此更新语句将是：

UPDATE [dbo].[SACL]
SET 
[Event]=
LEFT(Headline,CHARINDEX(' by ',Headline)),
[Username] = 
SUBSTRING(
    Headline,
    CHARINDEX(' by ',Headline)+4,
    CHARINDEX(' on ',Headline) - CHARINDEX(' by ',Headline)-4
    ),
[Path] = 
SUBSTRING(
    Headline,
    CHARINDEX(' by ',Headline)+4,
    CHARINDEX(' on ',Headline) - CHARINDEX(' by ',Headline)-4
    ),

[FileName]=
REVERSE(LEFT(
    REVERSE(Headline),
    CHARINDEX('/',REVERSE(Headline))-1)
    )
WHERE 
-- Only update rows that fit the pattern
Headline LIKE '% by %\% on %/%'
-- Only update rows that haven't been processed
AND ([Event] IS NULL  OR [Event]  = '')

请注意：

我不知道哪个字段包含完整的字符串。我假设Headline
我建议您对Event字段的内容进行标准化。 NULL =未处理，空白表示其他内容（已处理但无法解决）
我还建议你添加一个类型为datetime的ImportedDate和ProcessedDate字段。 ImportedDate的{{1}}为default，这意味着这是导入记录的日期时间。应该将UPDATE语句更新为ProcessedDate为GETDATE()，这会告诉您记录何时更新。

VB.Net将CSV文件导入SQL数据库

3 个答案: