Question

您可以为powershell中的变量分配的字符串大小是否有限制，或者对SQL INSERT查询中发送的文本大小有任何限制？

我有一个大的CSV文件进入PowerShell并通过foreach循环中的字符串构造我为每一行生成SQL INSERT查询。生成的INSERT查询; INSERT查询;超过大约4MB。

SQL服务器有一个完美的模式来接收数据，但是，当发送4MB INSERT查询集合（每个由;分隔）时，我得到一个错误，在我看来就像长4MB的插入集查询被某种方式截断了。我想我已达到某种限制。

有没有办法解决这个问题（以编程方式编程）或增加可接受的SQL INSERT查询集合的大小限制？

我的代码正在使用System.Data.SqlClient.SqlConnection和System.Data.sqlclient.SqlCommand。

较小的数据集工作正常，但较大的数据集会产生如下例所示的错误。每个不同的数据集都会发出不同的“语法不正确”指示符。

Exception calling "ExecuteNonQuery" with "0" argument(s): "Incorrect syntax
near '('."
At C:\Users\stuart\Desktop\git\ADStfL\WorkInProgress.ps1:211 char:3
+         $SQLCommand.executenonquery()
+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : SqlException

Answer 1

根据我的经验，执行此操作的最佳方法是将CSV加载到DataTable中，然后使用SQLBulkCopy。

$ErrorActionPreference = 'Stop';

$Csv = Import-Csv -Path $FileName;

$SqlServer = 'MyServer';
$SqlDatabase = 'MyDatabase';
$DestinationTableName = 'MyTable';

# Create Connection String
$SqlConnectionString = 'Data Source={0};Initial Catalog={1};Integrated Security=SSPI' -f $SqlServer, $SqlDatabase;


# Define your DataTable.  The column order of the DataTable must either match the table in the database, or 
# you must specify the column mapping in SqlBulkCopy.ColumnMapping.  If you have an IDENTITY column, it's a
# bit more complicated
$DataTable = New-Object -TypeName System.Data.DataTable -ArgumentList $DestinationTableName;

$NewColumn = $DataTable.Columns.Add('Id',[System.Int32]);
$NewColumn.AllowDBNull = $false;

$NewColumn = $DataTable.Columns.Add('IntegerField',[System.Int32]);
$NewColumn.AllowDBNull = $false;

$NewColumn = $DataTable.Columns.Add('DecimalField',[System.Decimal]);
$NewColumn.AllowDBNull = $false;

$NewColumn = $DataTable.Columns.Add('VarCharField',[System.String]);
$NewColumn.MaxLength = 50;

$NewColumn = $DataTable.Columns.Add('DateTimeField',[System.DateTime]);
$NewColumn.AllowDBNull = $false;


# Populate your datatable from the CSV file
# You may find that you need to type cast some of the fields.
$Csv | ForEach-Object {
    $NewRow = $DataTable.NewRow();
    $NewRow['Id'] = $_.Id;
    $NewRow['IntegerField'] = $_.IntegerField;
    $NewRow['DecimalField'] = $_.DecimalFiled;
    $NewRow['StringField'] = $_.StringField1;
    $NewRow['DateTimeField'] = $_.DateTimeField1;

    $DataTable.Rows.Add($NewRow);
}

# Create Connection
$SqlConnection = New-Object -TypeName System.Data.SqlClient.SqlConnection -ArgumentList $SqlConnectionString;

# Open Connection
$SqlConnection.Open();

# Start Transaction
$SqlTransaction = $SqlConnection.BeginTransaction();

# Double check the possible options at https://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlbulkcopyoptions(v=vs.110).aspx
# If you need multiple then -bor them together
$SqlBulkCopyOptions = [System.Data.SqlClient.SqlBulkCopyOptions]::CheckConstraints;

# Create SqlBulkCopy class
$SqlBulkCopy = New-Object -TypeName System.Data.SqlClient.SqlBulkCopy -ArgumentList $SqlConnection, $SqlBulkCopyOptions, $SqlTransaction;

# Specify destination table
$SqlBulkCopy.DestinationTableName = $DestinationTableName;

# Do the insert; rollback on error
try {
    $SqlBulkCopy.WriteToServer($DataTable);
    $SqlTransaction.Commit();
}
catch {
    # Roll back transaction and rethrow error
    $SqlTransaction.Rollback();
    throw ($_);
}
finally {
    $SqlConnection.Close();
    $SqlConnection.Dispose();
}

另一种方法是使用SQLCommand并逐行执行：

$ErrorActionPreference = 'Stop';

$Csv = Import-Csv -Path $FileName;

$SqlServer = 'MyServer';
$SqlDatabase = 'MyDatabase';

# Create Connection String
$SqlConnectionString = 'Data Source={0};Initial Catalog={1};Integrated Security=SSPI' -f $SqlServer, $SqlDatabase;

# Create Connection
$SqlConnection = New-Object -TypeName System.Data.SqlClient.SqlConnection -ArgumentList $SqlConnectionString;

# Create Command
$InsertCommandText = 'INSERT INTO DestinationTable (Id, IntegerField, DecimalField, StringField, DateTimeField) VALUES (@Id, @IntegerField, @DecimalField, @StringField, @DateTimeField)';
$InsertCommand = New-Object -TypeName System.Data.SqlClient.SqlCommand -ArgumentList $SqlConnection;

[void]$InsertCommand.Parameters.Add('@Id', [System.Data.SqlDbType]::Int);
[void]$InsertCommand.Parameters.Add('@IntegerField', [System.Data.SqlDbType]::Int);
[void]$InsertCommand.Parameters.Add('@DecimalField', [System.Data.SqlDbType]::Decimal);
[void]$InsertCommand.Parameters.Add('@StringField', [System.Data.SqlDbType]::VarChar,50);
[void]$InsertCommand.Parameters.Add('@DateTimeField', [System.Data.SqlDbType]::DateTime);

# Open connection and start transaction
$SqlConnection.Open()
$SqlTransaction = $SqlConnection.BeginTransaction();
$InsertCommand.Transaction = $SqlTransaction;
$RowsInserted = 0;  

try {
    $line = 0;
    $Csv | ForEach-Object {
        $line++;

        # Specify parameter values
        $InsertCommand.Parameters['@Id'].Value = $_.Id;
        $InsertCommand.Parameters['@IntegerField'].Value  = $_.IntegerField;
        $InsertCommand.Parameters['@DecimalField'].Value = $_.DecimalField;
        $InsertCommand.Parameters['@StringField'].Value = $_.StringField;
        $InsertCommand.Parameters['@DateTimeField'].Value = $_.DateTimeField;

        $RowsInserted += $InsertCommand.ExecuteNonQuery();

        # Clear parameter values
        $InsertCommand.Parameters | ForEach-Object { $_.Value = $null };
    }
    $SqlTransaction.Commit();
    Write-Output "Rows affected: $RowsInserted";
}
catch {
    # Roll back transaction and rethrow error
    $SqlTransaction.Rollback();
    Write-Error "Error on line $line" -ErrorAction Continue;
    throw ($_);
}
finally {
    $SqlConnection.Close();
    $SqlConnection.Dispose();
}

编辑：哦，我忘记了一个重点。如果需要在数据库中将字段的值设置为null，则需要将其值设置为[System.DBNull]::Value，而不是$null。

PowerShell管道/ SQL插入查询数据限制（并增加它们？）

1 个答案: