Question

我编写了一个PowerShell脚本，它遍历大量的IIS W3C日志文件并将值插入到MSSQL数据库中。

Set-Variable -Name "UnprocessedDir" -Value "X:\files" -Description "Folder for unprocessed log files" Scope Script
Set-Variable -Name "InputObject" -Value (New-Object -comObject MSUtil.LogQuery.IISW3CInputFormat) -Description "Log Parser input COM object" -Scope Script
Set-Variable -Name "OutputObject" -Value (New-Object -comObject MSUtil.LogQuery.SQLOutputFormat) -Description "Log Parser output COM object" -Scope Script
    $OutputObject.clearTable = $false
    $OutputObject.createTable = $false
    $OutputObject.database = "Database_Name"
    $OutputObject.driver = "SQL Server"
    $OutputObject.dsn = "DSN_Name"
    $OutputObject.fixColNames = $true
    $OutputObject.ignoreIdCols = $true
    $OutputObject.ignoreMinWarns = $true
    $OutputObject.maxStrFieldLen = 511
    $OutputObject.oConnString = $null
    $OutputObject.password = $null
    $OutputObject.server = "sqlserver.domain.com\INSTANCENAME"
    $OutputObject.transactionRowCount = 5000
    $OutputObject.username = $null

    Set-Variable -Name "IISLogs" -Value @(Get-ChildItem -Path $UnprocessedDir -Recurse -File) -Description "Array of files to be imported into SQL" -Scope Script
    Set-Variable -Name "LPComObj" -Value (New-Object -com MSUtil.LogQuery) -Description "COM Object used to import Log Parser records into MSSQL" -Scope Script
    Write-Output "$(Get-ISOTimeStamp) Beginning SQL import. $($IISLogs.Count) Files to be imported"

    $IISLogs | ForEach-Object { $loop = 0 } {
        Set-Variable -Name "SubDir" -Value $(($_.FullName).Split('\')[-2]) -Description "Subdirectory where log file is located" -Scope Script
        Set-Variable -Name "LogType" -Value $(($_.FullName).Split('\')[-3]) -Description "Type of log being imported" -Scope Script
        Set-Variable -Name "ServerName" -Value $(($_.FullName).Split('\')[-4]) -Description "ServerName of file being imported" -Scope Script
        Set-Variable -Name "LPQuery" -Description "Query to use in Log Parser" -Scope Script -Value @"
SELECT
-- FIELDS   LogFilename,LogRow,date,time,c-ip,cs-username,s-sitename,s-computername,s-ip,s-port,cs-method,cs-uri-stem,cs-uri-query,sc-status,sc-substatus,sc-win32-status,sc-bytes,cs-bytes,time-taken,cs-version,cs-host,cs(User-Agent),cs(Cookie),cs(Referer),s-event,s-process-type,s-user-time,s-kernel-time,s-page-faults,s-total-procs,s-active-procs,s-stopped-procs
-- STANDARD FIELDS date,time,s-ip,cs-method,cs-uri-stem,cs-uri-query,s-port,cs-username,c-ip,cs(User-Agent),cs(Referer),sc-status,sc-substatus,sc-win32-status,time-taken

    '$($ServerName)' as [servername],
    '$($_.Name)' as [filename],
    LogRow AS [row],
    '$($LogType)' as [logtype],
    TO_TIMESTAMP(date,time) AS [timestamp],
    [s-ip],
    [cs-method],
    [cs-uri-stem],
    [cs-uri-query],
    TO_INT([s-port]),
    [cs-username],
    [c-ip],
    [cs(User-Agent)],
    [cs(Referer)],
    TO_INT([sc-status]),
    TO_INT([sc-substatus]),
    TO_INT([sc-win32-status]),
    TO_INT([time-taken]),
    0 AS lock

INTO IIS_W3C

FROM '$($_.FullName)'
"@
        Set-Variable -Name "LPResult" -Value ($LPComObj.ExecuteBatch($LPQuery, $InputObject, $OutputObject)) -Description "IIS Log File imported into SQL" -Scope Script

        If ($LPResult -eq $false)
        {

            Write-Output "$(Get-ISOTimeStamp) Data imported from `"$($_.FullName)`""
            Set-Variable -Name "loop" -Value ($loop + 1) -Description "Increase loop iteration Count" -Scope Script
        }
        Else
        {
            Write-Output "$(Get-ISOTimeStamp) Log Parser returned errors importing `"$($_.FullName)`""
            Throw "$(Get-ISOTimeStamp) Log Parser returned errors importing `"$($_.FullName)`""
        }
    }

我导入的日志数量是数万;上面的代码非常适用于几百个文件，但几个小时后，它就会崩溃。从我所知，看起来ForEach-Object循环的每次迭代都会创建一个新的SQL TCP连接，该连接在循环结束时不会终止。

我尝试在循环内外创建$LPComObj。我试过了Remove-Variable。我尝试了一些通用命令，例如$LPComObj.Close()，.Remove()，.Quit()等。MSUtil.LogQuery方法本身似乎不包含任何关闭SQL TCP的方法连接，并且在脚本运行时，我可以看到越来越多的TCP连接堆积起来。我尝试使用[System.Runtime.Interopservices.Marshal]::释放/删除COM对象，但没有一个关闭TCP连接。即使关闭PowerShell会话也不会终止连接。

我能做到这一点的唯一方法就是找到dllhost.exe＆＃34;正在使用端口并将其终止的进程。但是从脚本中，没有一种干净的方法来获取有问题的dllhost.exe进程的PID。（尝试修改Get-Process | Stop-Process的某些变体可能会有效，但会在执行脚本时添加很多时间。）

我可以通过哪些其他方式解决此问题？

Powershell MSUtil.LogQuery没有关闭SQL连接

0 个答案: