BULK插入失败,文件大小不同

时间:2017-02-13 07:44:58

标签: sql-server vb.net sql-server-2008 stored-procedures

我遇到这个问题,批量插入失败,文件大小不同,具体来说,第一个文件包含433​​行,而第二个文件只有2行。但是当我自己插入2行文件时,没有错误。

我将此作为我的批量插入声明

set @sql =   'BULK INSERT #temptable
    FROM '''+@location+'''
    WITH
    (
    FIELDTERMINATOR = '','',  
    ROWTERMINATOR = ''\n'',
    MAXERRORS = 100,
    TABLOCK,
    FIRSTROW = 2
    )'

上述语句包含在存储过程中。

有关详细信息,我将把我的存储过程放在这里。

USE [THERMOWAVE]
GO
/****** Object:  StoredProcedure [dbo].[UploadOmronData]    Script Date: 02/13/2017 13:48:54 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER procedure [dbo].[UploadOmronData] 

@location as nvarchar(max)

as

begin
declare @sql as nvarchar(max)
-- give @location data from VB.net
 --declare @location as varchar(max)
 --set @location = 'C:\Users\jsumgo\Documents\Visual Studio 2010\Projects\TestUploader\TestUploader\bin\Debug\12320161428420dataUpload.csv'
-- Joshua Magsino 11/28/2016
-- TSql to upload dynamic column data source

--before uploading, make sure that datasource table columns are named equal to database table column
--Check if data is to be stored as reference or could be destroyed
--Check database to store and columns that would be affected

--set configurations to enable Ad Hoc Distributed Queries
/*
sp_configure 'show advanced options', 1;  
RECONFIGURE;
GO 
sp_configure 'Ad Hoc Distributed Queries', 1;  
RECONFIGURE;  
GO  
*/
-- configure SQL to execute Ad Hoc Queries using MSOFFICE 2010
/*
USE [master] 
GO 
EXEC master.dbo.sp_MSset_oledb_prop N'Microsoft.ACE.OLEDB.12.0', N'AllowInProcess', 1 
GO 
EXEC master.dbo.sp_MSset_oledb_prop N'Microsoft.ACE.OLEDB.12.0', N'DynamicParameters', 1 
GO  
*/


-- fill temptable with data

---------------------------------------------------------------------------------------------------------------
--drop table #temptable
--create blank temptable
create table #temptable
(
[Number] Varchar(20)
)

--declare @sql as varchar(max)
declare @cursor CURSOR
declare @colname as nvarchar(30)
SET @cursor = CURSOR FOR 
(Select colname from dbo.sorting)

OPEN @cursor

FETCH NEXT

FROM @cursor INTO @colname
WHILE @@FETCH_STATUS = 0
BEGIN
-- add from column 2 to end
if @colname <> 'Number'
    begin
        set @sql = 'ALTER TABLE #temptable ADD ['+@colName+'] NVARCHAR(max) NULL'
        EXEC sp_executesql @sql, N'@colname nvarchar(max)', @colname 
    end


FETCH NEXT
FROM @cursor INTO @colName
END

CLOSE @cursor
DEALLOCATE @cursor

---------------------------------------------------------------------------------------------------------------

--declare @location as varchar(max)
--set @location = 'D:\Omron Data\Data\sample\asdasdasdasdasd - Copy.csv'
--declare @sql as varchar(max)
--Insert Data from CSV file to Temptable
set @sql =   'BULK INSERT #temptable
    FROM '''+@location+'''
    WITH
    (
    FIELDTERMINATOR = '','',  
    ROWTERMINATOR = ''\n'',
    MAXERRORS = 100,
    TABLOCK,
    FIRSTROW = 2
    )'

EXEC sp_executesql @sql, N'@location as nvarchar(max)', @location 

--Insert Statement
--If the location of the new column is in the last column of the CSV file, then direct bulk insert
--safe solution is to insert based on the column name of each table
--cursor is needed

DECLARE @DBcursor CURSOR
DECLARE @DBColname as varchar(30)
DECLARE @DBCOL as nvarchar(max)

--Get Table Names from dbo.sorting
--since all table names are inserted from dbo.sorting, get column names from dbo.sorting
SET @DBcursor = CURSOR FOR 
(Select colname from dbo.sorting)

OPEN @DBcursor

FETCH NEXT
FROM @DBcursor INTO @DBColname
WHILE @@FETCH_STATUS = 0
BEGIN

--assemble columns
set @DBCOL = (isnull(@DBCOL,''))+'['+ @DBColname +'],'

FETCH NEXT
FROM @DBcursor INTO @DBColname
END

CLOSE @DBcursor
DEALLOCATE @DBcursor
--remove last (,) from columns
set @DBCOL =LEFT(@DBCOL, LEN(@DBCOL) - 1)
--insert data from temptable to DataUpload



set @sql = 'insert into DataToUpload  ('+@DBCOL+')  select '+@DBCOL+' from #tempTable'
EXEC sp_executesql @sql,N'@DBCOL NVARCHAR(max)', @DBCOL

--Put BatchID for DataUpload
--BatchId format,Batch + Number(000) + year + month + day
declare @number as nvarchar(max)
set @number = RIGHT('000' +CAST((select  ISNULL(max(case when BatchID = null or BatchID = '' then '000' else right(SUBSTRING(BatchID,1,8),3)+ 1 end),'000') from DataToUpload)AS VARCHAR(3)),3)
set @sql = 'update DataToUpload set BatchID = ''Batch'+cast(@number as varchar(max))+cast(year(getdate()) as varchar(4))+''+cast(MONTH(getdate())as varchar(2))+''+cast(DAY(getdate())as varchar(2)) +''' where BatchID is null'

EXEC sp_executesql @sql,N'@number NVARCHAR(max)', @number

--select * from DataToUpload order by BAtchID,Number asc
--select * from dbo.Sorting
--select * from temptable
--delete from DataToUpload

--clear dbo.sorting
delete from dbo.Sorting

--remove temptable
drop table #temptable

DBCC FREEPROCCACHE WITH NO_INFOMSGS;  


end

然后在vb.net系统中通过此处触发存储过程的命令。

 Sub UploadOmronData(ByVal location As String)
        'SQL Stored Procedure -- Save data from CSV file to SQL
        Dim sqlcom As New SqlClient.SqlCommand
        Try
            With sqlcom
                conn.Open()
                .Connection = conn
                .CommandTimeout = 100
                .CommandText = "dbo.UploadOmronData"
                .CommandType = CommandType.StoredProcedure
                .Parameters.AddWithValue("@location", location)
                .ExecuteReader()
            End With

        Catch ex As Exception
            MsgBox(ex.Message + " " + location)
            GC.Collect()
            sqlcom.Dispose()
            frmMain.Close()
        End Try
        conn.Close()
        sqlcom.Dispose()
        GC.Collect()
    End Sub

我收到的错误消息:

Bulk load data conversion error (truncation) for row 2, column 1 (Number).

Bulk load data conversion error (truncation) for row 3, column 1 (Number). 

添加信息

这是dbo.DataToUpload

的表结构
CREATE TABLE [dbo].[DataToUpload](
    [Number] [int] NULL,
    [Date&Time] [nvarchar](max) NULL,
    [ms] [nvarchar](50) NULL,
    [CH34] [nvarchar](50) NULL,
    [CH35] [nvarchar](50) NULL,
    [CH36] [nvarchar](50) NULL,
    [CH37] [nvarchar](50) NULL,
    [CH38] [nvarchar](50) NULL,
    [CH39] [nvarchar](50) NULL,
    [Alarm1-10] [nvarchar](50) NULL,
    [Alarm11-20] [nvarchar](50) NULL,
    [Alarm21-30] [nvarchar](50) NULL,
    [Alarm31-40] [nvarchar](50) NULL,
    [AlarmOut] [nvarchar](50) NULL,
    [BatchID] [nvarchar](50) NULL,
    [CH31] [nvarchar](20) NULL,
    [CH32] [nvarchar](20) NULL,
    [CH33] [nvarchar](20) NULL
) ON [PRIMARY]

以及我要插入的行

Number,Date&Time,ms,CH34,CH35,CH36,CH37,CH38,CH39,Alarm1-10,Alarm11-20,Alarm21-30,Alarm31-40,AlarmOut
1,2016-08-08 16:23:16,000,+61.2,+64.0,+35.4,+94.4,+185.4,+151.2,LLLLLLLLLL,LLLLLLLLLL,LLLLLLLLLL,LLLLLLLLLL,LLLL
2,2016-08-08 16:23:26,000,+61.1,+64.4,+35.4,+94.3,+185.4,+151.2,LLLLLLLLLL,LLLLLLLLLL,LLLLLLLLLL,LLLLLLLLLL,LLLL

如果有帮助,我创建的系统是后台工作者,因为我必须始终运行代码。

2 个答案:

答案 0 :(得分:1)

我认为原因是您要求BULK INSERT加载14列数据,但是您要导入#TempTable并且没有足够的字段它

您的代码会创建一个#tempTable表,并使用CURSOR向其中添加额外的字段...但我认为最终结果 isn&#39; t 一张包含14个字段的表格。

BULK INSERT #temptable命令运行之前,#temptable表中有多少个字段?

例如,如果您在此表中只有一个字段,那么当您运行批量插入时,它会坚持尝试将整行导入到该字段中,并使用所有逗号完成但是,当然,它不适合20个字符的字符串。

CREATE TABLE #temptable
(
    [Number] VARCHAR(20)
)

解决方案......只需更改临时表以获得正确的列数,并且BULK INSERT 正常工作:

CREATE TABLE #temptable
(
    [Number] NVARCHAR(200),
    [Datey] datetime,
    [ms] NVARCHAR(200),
    [CH34] NVARCHAR(200),
    [CH35] NVARCHAR(200),
    [CH36] NVARCHAR(200),
    [CH37] NVARCHAR(200),
    [CH38] NVARCHAR(200),
    [CH39] NVARCHAR(200),
    [Alarm110] NVARCHAR(200),
    [Alarm1120] NVARCHAR(200),
    [Alarm2130] NVARCHAR(200),
    [Alarm3140] NVARCHAR(200),
    [AlarmOut] NVARCHAR(200)
)
祝你好运!

P.S。丢失的一块拼图......你的dbo.sorting表中有哪些数据?这将指示哪些字段最终会出现在您的临时表中,但您无法解释如何填充此表,或者它包含哪些数据。

答案 1 :(得分:0)

属性

[BatchID] [nvarchar](50) NULL,
[CH31] [nvarchar](20) NULL,
[CH32] [nvarchar](20) NULL,
[CH33] [nvarchar](20) NULL
您的文件中缺少

。使用BULK_INSERT时,列必须完美匹配。修复列数,它应该修复错误。