Question

我有一个powershell脚本，它从特定目录开始递归地写入每个文件及其属性。这可以工作，但目录可能有多达1,000,000个文件。我想要做的是每次交易1000个插入批处理它们。这是最初的PS：

glfwOpenWindow( 512, 512, 0, 0, 0, 0, 0, 0, GLFW_WINDOW )
                                      ^ no depth bits

我的想法是实现某种计数器，它会一直追加插入，直到它达到1000，然后跳出循环并执行。我无法弄清楚当前的设置如何在1000处批处理，执行然后使用get-childitem循环进行备份。

Answer 1

$server = ""
$Database = ""
$Path = "C:\Test"

$Connection = New-Object System.Data.SQLClient.SQLConnection
$Connection.ConnectionString = "server='$Server';database='$Database';trusted_connection=true;"
$Connection.Open()
$Command = New-Object System.Data.SQLClient.SQLCommand
$Command.Connection = $Connection

# new variables to handle batching
$batchcounter=0
$batchsize=1000
$sqlValues = New-Object Collections.ArrayList

foreach($file in Get-ChildItem -Verbose -Recurse -Path $Path | Select-Object  Name,Length,Mode,  Directory,CreationTime, LastAccessTime, LastWriteTime) {
    $fileName = $file.Name
    $fileSize = ([int]$file.Length)
    $fileMode = $file.Mode
    $fileDirectory = $file.Directory
    $fileCreationTime = [datetime]$file.CreationTime
    $fileLastAccessTime = [datetime]$file.LastAccessTime
    $fileLastWriteTime = [datetime]$file.LastWriteTime

    $sqlValues.Add("('$fileName', '$fileSize', '$fileMode', '$fileDirectory', '$fileCreationTime', '$fileLastAccessTime', '$fileLastWriteTime')")

    $batchcounter++

    # if the counter hits batchsize, run the insert, using lots of:
    # insert into table
    # values (1,2,3)
    #      , (4,5,6)
    #      , (7,8,9)

    if ($batchcounter % $batchsize -eq 0) {
        $sql = "insert TestPowerShell values {0}" -f ($sqlValues.ToArray() -join "`r`n,")
        $Command.CommandText = $sql
        Write-Host $sql
        $Command.ExecuteNonQuery()
        $sqlValues.Clear()
    }
}

# catch any remaining files
if ($batchcounter -gt 0) {
    $sql = "insert TestPowerShell values {0}" -f ($sqlValues.ToArray() -join "`r`n,")
    $Command.CommandText = $sql
    Write-Host $sql
    $Command.ExecuteNonQuery()
    $sqlValues.Clear()
}

$Connection.Close()

Answer 2

这样的事情应该做：

function Execute-SqlQuery($query){
    Write-Host "Executing query:" 
    Write-Host $query;
}

$data = @(1,2,3,4,5,6,7,8,9,10,11);
$batchSize = 2;
$counter = 0;
$sql = "";

foreach($item in $data){

    if($counter -eq $batchSize){
        Execute-SqlQuery $sql;
        $counter = 0;
        $sql = "";
    }

    $sql += "insert into myTable(id) values($item) `n";
    $counter += 1;    
}

Execute-SqlQuery $sql;

Answer 3

对于任何感兴趣的人 - 这是一种方法：

function WriteBatch {
    echo $sql
    $Command.CommandText = $sql
        $Command.ExecuteNonQuery()
}

$server = ""
$Database = ""
$Path = ""
$Counter = 0

$Connection = New-Object System.Data.SQLClient.SQLConnection
$Connection.ConnectionString = "server='$Server';database='$Database';trusted_connection=true;"
$Connection.Open()
$Command = New-Object System.Data.SQLClient.SQLCommand
$Command.Connection = $Connection
[string]$sql =     "
                begin 
                    insert into TestPowerShell(NameString, FileSize, Mode, Directory, CreationTime, LastAccessTime, LastWriteTime)
                    values " 

foreach($file in Get-ChildItem -Verbose -Recurse -Path $Path | Select-Object  Name, Length, Mode, Directory, CreationTime, LastAccessTime, LastWriteTime) {
    $fileName = $file.Name
    $fileSize = ([int]$file.Length)
    $fileMode = $file.Mode
    $fileDirectory = $file.Directory
    $fileCreationTime = [datetime]$file.CreationTime
    $fileLastAccessTime = [datetime]$file.LastAccessTime
    $fileLastWriteTime = [datetime]$file.LastWriteTime
    $sql = $sql + "('$fileName', '$fileSize', '$fileMode', '$fileDirectory', '$fileCreationTime', '$fileLastAccessTime', '$fileLastWriteTime'),"
    $sql += "`n"
    $Counter++

    If($Counter -eq 900) {
        $sql = $sql.Trim().Trim(',')
        $sql = $sql + " End"
        WriteBatch
        $Counter = 0
        $sql = "
                begin 
                    insert into TestPowerShell(NameString, FileSize, Mode, Directory, CreationTime, LastAccessTime, LastWriteTime)
                    values " 
    }

}

if ($Counter -gt 0){
    $sql = $sql.Trim().Trim(',')
    $sql = $sql + " End"
    WriteBatch
}

$Connection.Close()

构建批量插入语句powershell到sql

3 个答案: