Get-ChildItem -Recurse抛出System.OutOfMemoryException

时间:2016-10-12 23:42:13

标签: function powershell memory memory-management

我在一个大目录上运行此函数:

定义

# (searches for all ips within files in the current directory recursively)

function searchips
{
    param(
        [Parameter(Mandatory=$false)][string]$dir = $(pwd)
    )

    ls -Recurse -Force `
    | Select-String -Pattern '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' -AllMatches `
    | ? { 
        $matches = ($_.Matches | Select-Object -Unique)
        return $matches.Count -gt 1 -or $matches[0].Value -ne '127.0.0.1' 
    } `
    | select Path,Matches,FileName,LineNumber,Line `
    | Format-Table -AutoSize `
    | Out-String -Width 4096
}

呼叫

PS C:\path\to\huge> searchips hugeDirectory >> outfile.txt

但我每次都会收到此错误:

out-lineoutput : Exception of type 'System.OutOfMemoryException' was thrown.
At C:\Users\myUserName\Documents\WindowsPowerShell\profile.ps1:73 char:2
+     ls -Recurse `
+     ~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [out-lineoutput], OutOfMemoryException
    + FullyQualifiedErrorId : System.OutOfMemoryException,Microsoft.PowerShell.Commands.OutLineOutputCommand

目前我的PS内存设置如下:

   WSManConfig: Microsoft.WSMan.Management\WSMan::localhost\Shell

Name                      Value         Type
----                      -----         ----
AllowRemoteShellAccess    true          System.String
IdleTimeout               7200000       System.String
MaxConcurrentUsers        10            System.String
MaxShellRunTime           2147483647    System.String
MaxProcessesPerShell      25            System.String
MaxMemoryPerShellMB       9000000       System.String
MaxShellsPerUser          30            System.String

有什么想法吗?

1 个答案:

答案 0 :(得分:3)

问题是由Out-String引起的。 cmdlet将其输入合并为单个字符串(在将该字符串返回给调用者之前)。为此,它必须收集内存中的所有输出。有点显而易见,现在我想起来了。

我建议使用ConvertTo-Csv代替Format-Table | Out-String。它避免了内存耗尽,并且使用输出进行进一步处理要容易得多。

function Find-IPAddresses {
    Param(
        [Parameter(Mandatory=$false)]
        [ValidateScript({Test-Path -LiteralPath $_})]
        [string]$dir = $PWD.Path
    )

    Get-ChildItem $dir -Recurse -Force |
        Select-String -Pattern '\d{1,3}(?:\.\d{1,3}){3}' -AllMatches |
        Where-Object {
            $matches = ($_.Matches | Select-Object -Unique)
            $matches.Count -gt 1 -or $matches[0].Value -ne '127.0.0.1'
        } |
        Select-Object Path, Matches, FileName, LineNumber, Line |
        ConvertTo-Csv -NoType
}

Find-IPAddresses 'C:\some\folder' > 'outfile.csv'

或根本不回复文字输出。只需返回对象列表,并在调用函数时进行所有格式化/输出,并知道要对数据执行的操作:

function Find-IPAddresses {
    Param(
        [Parameter(Mandatory=$false)]
        [ValidateScript({Test-Path -LiteralPath $_})]
        [string]$dir = $PWD.Path
    )

    Get-ChildItem $dir -Recurse -Force |
        Select-String -Pattern '\d{1,3}(?:\.\d{1,3}){3}' -AllMatches |
        Where-Object {
            $matches = ($_.Matches | Select-Object -Unique)
            $matches.Count -gt 1 -or $matches[0].Value -ne '127.0.0.1'
        } |
        Select-Object Path, Matches, FileName, LineNumber, Line
}

Find-IPAddresses 'C:\some\folder' | Export-Csv 'outfile.csv' -NoType