Question

我还习惯使用SQL，所以在开始使用存储过程之前，我想先了解如何有效地使用BULK INSERT。

我需要合并50个以上的csv文件并将它们转储到SQL表中。问题是，我希望能够区分每条记录（例如，每条记录属于某个csv文件，我将通过文件名识别）。

以下是我想要的一个小例子：

CREATE TABLE ResultsDump
(
    PC FLOAT,
    Amp VARCHAR(50),
    RCS VARCHAR(50),
    CW VARCHAR(50),
    State0 VARCHAR(50),
    State1 VARCHAR(50),
)
BULK INSERT ResultsDump
    FROM 'c:\distance1000_7_13_2010_1_13PM_Avery DennisonAD_2300008_10S_Lock.csv' 
    WITH 
    ( 
        FIRSTROW = 2, 
        MAXERRORS = 0, 
        FIELDTERMINATOR = ',', 
        ROWTERMINATOR = '\n'
    )
BULK INSERT ResultsDump
    FROM 'c:\distance1000_7_13_2010_2_27PM_Avery DennisonAD_2300009_10S_Lock.csv' 
    WITH 
    ( 
        FIRSTROW = 2, 
        MAXERRORS = 0, 
        FIELDTERMINATOR = ',', 
        ROWTERMINATOR = '\n'
    )
BULK INSERT ResultsDump
    FROM 'C:\distance1000_7_13_2010_2_58PM_Avery DennisonAD_230000A_10S_Lock.csv' 
    WITH 
    ( 
        FIRSTROW = 2, 
        MAXERRORS = 0, 
        FIELDTERMINATOR = ',', 
        ROWTERMINATOR = '\n'
    )
BULK INSERT ResultsDump
    FROM 'c:\distance1000_7_13_2010_3_21PM_Avery DennisonAD_230000B_10S_Lock.csv' 
    WITH 
    ( 
        FIRSTROW = 2, 
        MAXERRORS = 0, 
        FIELDTERMINATOR = ',', 
        ROWTERMINATOR = '\n'
    )
BULK INSERT ResultsDump
    FROM 'c:\distance1000_7_13_2010_3_41PM_Avery DennisonAD_230000C_10S_Lock.csv' 
    WITH 
    ( 
        FIRSTROW = 2, 
        MAXERRORS = 0, 
        FIELDTERMINATOR = ',', 
        ROWTERMINATOR = '\n'
    )

我知道这是一种低效的处理方式，但我非常想弄清楚如何在开始创建存储过程之前以我想要的格式在SQL表中手动转储一个文件。

在新表中，我想要这样的东西：

FileName,PC,Amp,RCS,CW,State0,State1
c:\distance1000_7_13_2010_1_13PM_Avery DennisonAD_2300008_10S_Lock.csv, ...
...
...
c:\distance1000_7_13_2010_2_27PM_Avery DennisonAD_2300009_10S_Lock.csv, ...
...
...
c:\distance1000_7_13_2010_2_58PM_Avery DennisonAD_230000A_10S_Lock.csv, ...
...
...

对特定功能的任何简单建议或推荐都会很棒！记住，我已经习惯了SQL，如果我能一次采取这一步，那就太好了，这就是为什么我要从这么简单的问题开始。

提前致谢！

Answer 1

您可以向ResultsDump表添加列FileName varchar(max)，使用新列创建表的视图，批量插入到视图中，并在每次插入后，为其仍具有其的列设置文件名默认值null：

CREATE TABLE dbo.ResultsDump
(
    PC FLOAT,
    Amp VARCHAR(50),
    RCS VARCHAR(50),
    CW VARCHAR(50),
    State0 VARCHAR(50),
    State1 VARCHAR(50),
)
GO

ALTER TABLE dbo.ResultsDump ADD [FileName] VARCHAR(300) NULL 
GO 

CREATE VIEW dbo.vw_ResultsDump AS
SELECT
    PC,
    Amp,
    RCS,
    CW,
    State0,
    State1
FROM
    ResultsDump
GO

BULK INSERT vw_ResultsDump
    FROM 'c:\distance1000_7_13_2010_1_13PM_Avery DennisonAD_2300008_10S_Lock.csv' 
    WITH 
    ( 
        FIRSTROW = 2, 
        MAXERRORS = 0, 
        FIELDTERMINATOR = ',', 
        ROWTERMINATOR = '\n'
    )

UPDATE dbo.ResultsDump 
SET [FileName] = 'c:\distance1000_7_13_2010_1_13PM_Avery DennisonAD_2300008_10S_Lock.csv'
WHERE [FileName] IS NULL 

BULK INSERT vw_ResultsDump
    FROM 'c:\distance1000_7_13_2010_2_27PM_Avery DennisonAD_2300009_10S_Lock.csv' 
    WITH 
    ( 
        FIRSTROW = 2, 
        MAXERRORS = 0, 
        FIELDTERMINATOR = ',', 
        ROWTERMINATOR = '\n'
    )

UPDATE dbo.ResultsDump 
SET [FileName] = 'distance1000_7_13_2010_2_27PM_Avery DennisonAD_2300009_10S_Lock.csv'
WHERE [FileName] IS NULL

Answer 2

试试这个，

                    ALTER PROCEDURE [dbo].[ReadandUpdateFileNames_SP]
            (
             @spRequestId NVARCHAR(50)
            ,@LoopCounter INT =0
            ,@MaxFIVId INT=0
            ,@spFileName NVARCHAR(100)=NULL)
            AS
            BEGIN
            SET NOCOUNT ON
                BEGIN TRY
                    BEGIN TRAN
                    -- To read filename's from the Request Id and store it in a temp table
                        DECLARE @cmd nvarchar(500)
                        SET @cmd = 'dir D:\Input\REQUEST-'+@spRequestId+' /b '
                        --PRINT @cmd
                        DECLARE  @DirOutput TABLE(
                                      ID INT IDENTITY
                                    , files varchar(500))

                        INSERT INTO @DirOutput 
                        EXEC master.dbo.xp_cmdshell @cmd
                        SELECT * FROM @DirOutput WHERE files IS NOT NULL ORDER BY ID

                        ----Read files by RequestId BEGIN

                        SELECT @LoopCounter = min(ID) , @MaxFIVId = max(ID)
                            FROM @DirOutput 

                        WHILE(@LoopCounter IS NOT NULL AND @LoopCounter<@MaxFIVId)
                        BEGIN
                        -- Create temp table to store FIVItems 
                            CREATE  TABLE Items_TEMP  
                            (
                                  ControlID NVARCHAR(50)
                                , UNRS_Code NUMERIC(18,0)
                                , UNRS_Code_S NUMERIC(18,0)
                                , Ordered_Quantity NUMERIC(18,3)
                                , Sent_Quantity NUMERIC(18,3)
                                , Accepted_Quantity NUMERIC(18,3)
                                , Unit_Food_Price NUMERIC(18,2)
                                , Total_Price NUMERIC(18,2)
                            )
                        SELECT @spFileName=files FROM @DirOutput WHERE ID=@LoopCounter
                            PRINT @LoopCounter
                            DECLARE @spControlId NVARCHAR(50)
                            SET @spControlId='FFO'+ Substring(@spFileName, 4, (len(@spFileName)-7)) 
                            --PRINT @spControlId
                            DECLARE @sqlCmd NVARCHAR(MAX)
                            SET @sqlCmd='BULK INSERT 
                            Items_TEMP
                            FROM ''D:\Input\REQUEST-'+@spRequestId+'\'+@spFileName+'''
                            WITH(
                                  FIELDTERMINATOR='',''
                                , ROWTERMINATOR=''\n''
                            )'
                            PRINT @sqlCmd
                            EXECUTE sp_executesql @sqlCmd
                            ---Add a new column to the table which is not present in the CSV
                            ALTER TABLE Items_TEMP ADD OrderId NUMERIC(18,0)NULL
                            UPDATE Items_TEMP SET ControlID=@spControlId,OrderId=(SELECT OrderId FROM dbo.Orders WHERE ControlID=@spControlId)
                            SELECT * FROM Items_TEMP

                            --DROP FIVItems_TEMP table once CSV output generated
                            DROP TABLE Items_TEMP


                            SET @LoopCounter=@LoopCounter+1
                        END
                    ----****END***
                    COMMIT TRAN
                END TRY
                BEGIN CATCH
                    ROLLBACK TRAN
                    PRINT 'ROLLBACK'
                    PRINT Error_Message()
                    SELECT ERROR_LINE() AS ErrorLine;
                END CATCH
            SET NOCOUNT OFF
            END

如何使用显示文件名的其他列进行BULK INSERT？

2 个答案: