Powershell GCI,以达到速度

时间:2013-02-25 19:24:27

标签: multithreading powershell

我需要一种方法来加速以下代码,以便在非常大的服务器上使用。我需要的只是全名param的列表。我想使用以下内容: http://newsqlblog.com/2012/05/22/concurrency-in-powershell-multi-threading-with-runspaces/

我正在阅读它并且看起来很简单,但是我无法绕过允许线程在现有池中创建新线程......以及如何传递该池。

任何帮助都会很棒,因为这种速度非常糟糕,而且我有更多的资源,然后我才会这样做。

Function Get-ChildItemToDepth {   
    Param(     
        [String]$Path = $PWD,    
        [int]$ToDepth = 255,     
        [Byte]$CurrentDepth = 0
    ) 
    if ($ToDepth -lt 0) {
        return get-item $path
    }
    $CurrentDepth++
    Get-ChildItem $Path | Where-Object { $_.PSIsContainer } | %{ $_ 
        If ($CurrentDepth -le $ToDepth) {
            Get-ChildItemToDepth -Path $_.FullName `
            -ToDepth $ToDepth -CurrentDepth $CurrentDepth
        } 
    }
}   

注意:我受限于v2.0

总结:我基本上需要所有文件夹的路径,最多可以达到$深度,尽可能快地进入数组。

MISSERABLE FAILURE ATTEMPT:

$func = `
{
    Param(     
        [String]$Path = $PWD,    
        [int]$ToDepth = 255,     
        [Byte]$CurrentDepth = 0,
        $p = $pool
    ) 
    if ($ToDepth -lt 0) {
        return $path
    }
    $CurrentDepth++
    $folders = Get-ChildItem $Path | Where-Object { $_.PSIsContainer } | select -expand FullName
    If ($CurrentDepth -le $ToDepth) {
        foreach ($path in $folders) {
            $pipeline  = [System.Management.Automation.PowerShell]::create()  
            $pipeline.RunspacePool = $pool 
            $pipeline.AddScript($func).AddArgument($path).AddArgument($ToDepth).AddArgument($CurrentDepth).AddArgument($pool)
            $AsyncHandle = $pipeline.BeginInvoke() 
            $folders += $pipeline.EndInvoke($AsyncHandle)  
            $pipeline.Dispose()  
        }
    }
    return $folders
}    

$path = "\\server\users-folder\"
$toDepth = 3

$pool = [RunspaceFactory]::CreateRunspacePool(1, 4) 
$pool.ApartmentState = "STA" 
$pool.Open() 
$pipeline  = [System.Management.Automation.PowerShell]::create()  
$pipeline.RunspacePool = $pool  
$pipeline.AddScript($func).AddArgument($path).AddArgument($toDepth).AddArgument($CurrentDepth).AddArgument($pool)
$AsyncHandle = $pipeline.BeginInvoke() 
$RESULTS = $pipeline.EndInvoke($AsyncHandle)  
$pipeline.Dispose()  
$pool.Close()

2 个答案:

答案 0 :(得分:1)

尝试在工作流中调用它 - 一个功能强大的本机PowerShell多线程解决方案,可直接使用PowerShell 3.0

Workflow Invoke-Function
{
Function Get-ChildItemToDepth {   
    Param(     
        [String]$Path = $PWD,    
        [int]$ToDepth = 255,     
        [Byte]$CurrentDepth = 0
    ) 
    if ($ToDepth -lt 0) {
        return $path
    }
    $CurrentDepth++
    Get-ChildItem $Path | Where-Object { $_.PSIsContainer } | %{ $_ 
        If ($CurrentDepth -le $ToDepth) {
            Get-ChildItemToDepth -Path $_.FullName `
            -ToDepth $ToDepth -CurrentDepth $CurrentDepth
        } 
    }
}  
Get-ChildItemToDepth 
}

这是一个相当混乱的实现,我只是将您的函数和调用工作流包装起来,因此您可以将其复制粘贴到会话中以便于测试。请告诉我这对您有何帮助。如果你想看到速度上的差异,我会将函数的结果传递给Measure

Get-ChildItemToDepth | Measure
Invoke-Function | Measure

用法(将工作流粘贴到会话中后)

Invoke-Function

答案 1 :(得分:0)

转到http://psasync.codeplex.com并下载psasync模块(文档http://newsqlblog.com/2012/12/17/psasync-module-multithreaded-powershell/)。然后......

Import-Module psasync

$AsyncPipelines = @()

# Allow 4 concurrent processes ... adjust up / down as hardware allows
$pool = Get-RunspacePool 4

# You can play with this to get better performance by going deeper into the directory structure to populate more dirs in the array
# gross generalization -> more dirs + more runspaces in $pool = faster performance

$rootDirs = Get-ChildItem c:\ | Where-Object{$_.PSIsContainer}

# Create a script block to run code per directory stored in $rootDirs
$ScriptBlock =  { ` 
    Param(     
        [String]$Path = $path
    ) 
    Get-ChildItem $path -recurse | Where-Object{$_.PSIsContainer}

}   

# Load up processes on the runspace pool
foreach($dir in $rootDirs) {
    $AsyncPipelines += Invoke-Async -RunspacePool $pool -ScriptBlock $ScriptBlock -Parameters $dir.FullName
}

# "Listen" to the pipelines and grab the results
$dirs = Receive-AsyncResults -Pipelines $AsyncPipelines

# PROFIT
$dirs | Foreach-Object{$_.FullName}

感谢阅读newsqlblog.com;)