使用运行空间进行多线程处理:方法调用会导致意外行为

时间:2014-02-14 09:55:41

标签: powershell

在文章Concurrency in PowerShell: Multi-threading with RunspacesMultithreading PowerShell Scripts之后,我一直在尝试使用运行空间进行多线程处理,并注意到一些我不理解的行为。

在下面的脚本中,脚本块$compute_block通过在返回之前休眠一秒来模拟昂贵的计算。产生了十个带有此计算的线程,然后脚本等待所有这些线程完成并打印结果。

该脚本可以使用-Case 1-Case 2运行。在案例1中,使用AddScript直接调用$compute_block。在案例2中,使用等效于$compute_block的方法创建对象,并为AddScript提供调用此方法的脚本块。

threading.ps1

Param([Int] $Case)

Set-StrictMode -Version 2

$start_date = Get-Date

Function Get-Timestamp() {
    Return ("{0:N3}" -f ((Get-Date) - $start_date).TotalSeconds)
}

Function Write-Timed([String] $str) {
    Write-Host "[$(Get-Timestamp)] $str"
}

$count = 10

$runspace_pool = [RunspaceFactory]::CreateRunspacePool(1, $count)
$runspace_pool.Open()

$compute_block = {
    Param($x)
    Start-Sleep 1
    Return $x
}

# Spawn jobs
$jobs = @()
ForEach($i In 0..($count-1)) {
    Write-Timed "Creating job #$i"
    $ps = [PowerShell]::Create()
    Switch($Case) {
        1 {
            $job = $ps.AddScript($compute_block).AddArgument($i)
        }

        2 {
            $object = New-Object PSObject `
                | Add-Member ScriptMethod Compute $compute_block -PassThru

            $job = $ps.AddScript({
                Param($o, $x)
                Return $o.Compute($x)
            }).AddArgument($object).AddArgument($i)
        }
    }
    $job.RunspacePool = $runspace_pool

    Write-Timed "Starting job #$i"
    $result = $job.BeginInvoke()

    Write-Timed "Creating record for job #$i"
    $record = New-Object PSObject -Property @{
        "Job" = $job;
        "Result" = $result;
    }

    Write-Timed "Adding record for job #$i to list"
    $jobs += $record
}

# Wait for all jobs to complete
While($true) {
    $running_count = @($jobs | Where-Object { -not $_.Result.IsCompleted }).Count
    Write-Timed "Waiting for $running_count/$count jobs"
    If(0 -eq $running_count) {
        Break
    }
    Start-Sleep 1
}

# Print results
$i = 0
ForEach($job In $jobs) {
    $result = $job.Job.EndInvoke($job.Result)
    $job.Job.Dispose()
    Write-Timed "Job #$i result: $result"
    $i++
}

$runspace_pool.Close()

结果非常不同(请注意时间戳):

案例1

PS C:\Users\Miranda\Documents> .\threading.ps1 -Case 1
[0.050] Creating job #0
[0.051] Starting job #0
[0.052] Creating record for job #0
[0.052] Adding record for job #0 to list
[0.053] Creating job #1
[0.053] Starting job #1
[0.056] Creating record for job #1
[0.057] Adding record for job #1 to list
[0.057] Creating job #2
[0.058] Starting job #2
[0.061] Creating record for job #2
[0.062] Adding record for job #2 to list
[0.062] Creating job #3
[0.063] Starting job #3
[0.066] Creating record for job #3
[0.066] Adding record for job #3 to list
[0.067] Creating job #4
[0.067] Starting job #4
[0.070] Creating record for job #4
[0.071] Adding record for job #4 to list
[0.071] Creating job #5
[0.072] Starting job #5
[0.075] Creating record for job #5
[0.076] Adding record for job #5 to list
[0.076] Creating job #6
[0.077] Starting job #6
[0.080] Creating record for job #6
[0.080] Adding record for job #6 to list
[0.081] Creating job #7
[0.081] Starting job #7
[0.084] Creating record for job #7
[0.085] Adding record for job #7 to list
[0.085] Creating job #8
[0.086] Starting job #8
[0.102] Creating record for job #8
[0.103] Adding record for job #8 to list
[0.104] Creating job #9
[0.104] Starting job #9
[0.114] Creating record for job #9
[0.115] Adding record for job #9 to list
[0.119] Waiting for 10/10 jobs
[1.120] Waiting for 0/10 jobs
[1.121] Job #0 result: 0
[1.122] Job #1 result: 1
[1.122] Job #2 result: 2
[1.123] Job #3 result: 3
[1.124] Job #4 result: 4
[1.124] Job #5 result: 5
[1.124] Job #6 result: 6
[1.125] Job #7 result: 7
[1.125] Job #8 result: 8
[1.126] Job #9 result: 9

案例2

PS C:\Users\Miranda\Documents> .\threading.ps1 -Case 2
[0.080] Creating job #0
[0.117] Starting job #0
[0.120] Creating record for job #0
[0.121] Adding record for job #0 to list
[1.126] Creating job #1
[1.128] Starting job #1
[1.129] Creating record for job #1
[2.130] Adding record for job #1 to list
[2.130] Creating job #2
[2.132] Starting job #2
[2.132] Creating record for job #2
[3.134] Adding record for job #2 to list
[3.135] Creating job #3
[3.136] Starting job #3
[3.137] Creating record for job #3
[4.137] Adding record for job #3 to list
[4.138] Creating job #4
[4.139] Starting job #4
[4.140] Creating record for job #4
[5.141] Adding record for job #4 to list
[5.142] Creating job #5
[5.143] Starting job #5
[5.144] Creating record for job #5
[6.144] Adding record for job #5 to list
[6.145] Creating job #6
[6.146] Starting job #6
[6.147] Creating record for job #6
[7.148] Adding record for job #6 to list
[7.149] Creating job #7
[7.150] Starting job #7
[7.151] Creating record for job #7
[8.152] Adding record for job #7 to list
[8.153] Creating job #8
[8.166] Starting job #8
[8.167] Creating record for job #8
[9.168] Adding record for job #8 to list
[9.169] Creating job #9
[9.170] Starting job #9
[9.171] Creating record for job #9
[10.172] Adding record for job #9 to list
[10.192] Waiting for 0/10 jobs
[10.206] Job #0 result: 0
[10.208] Job #1 result: 1
[10.209] Job #2 result: 2
[10.209] Job #3 result: 3
[10.209] Job #4 result: 4
[10.210] Job #5 result: 5
[10.211] Job #6 result: 6
[10.211] Job #7 result: 7
[10.212] Job #8 result: 8
[10.212] Job #9 result: 9

案例1的行为与我预期的一样 - 线程即时生成,脚本在完成之前在等待循环中花费一秒钟。

然而,在案例2中,所有并发似乎都丢失了。产卵循环的每次迭代都会阻塞,直到产生的线程结束,一旦达到等待循环,就没有什么可以等待了。为什么会这样?

编辑:

为了记录,我正在使用PS 3.0。正如Roman Kuzmin所说,PS 2.0中的运行案例2产生了一些非常奇怪的错误:

PS>.\threading.ps1 -Case 2
[0.060] Creating job #0
[0.060] Starting job #0
The '=' operator failed: Index was outside the bounds of the array..
At C:\Users\Miranda\Documents\threading.ps1:50 char:14
+     $result = <<<<  $job.BeginInvoke()
    + CategoryInfo          : InvalidOperation: (System.Manageme...hellAsyncResult:PowerShellAsyncResult) [], RuntimeE
   xception
    + FullyQualifiedErrorId : OperatorFailed

C:\Users\Miranda\Documents\threading.ps1 : Index was outside the bounds of the array.
At line:1 char:16
+ .\threading.ps1 <<<<  -Case 2
    + CategoryInfo          : NotSpecified: (:) [threading.ps1], IndexOutOfRangeException
    + FullyQualifiedErrorId : System.IndexOutOfRangeException,threading.ps1

1 个答案:

答案 0 :(得分:1)

我无法准确说明为什么PowerShell可以正常工作,但我可以告诉它是什么导致的 问题以及如何解决它。在案例2中脚本块 $compute_block用作10个对象/运行空间中的脚本方法 - 这是 罪魁祸首。如果我们制作和使用克隆的脚本块,即

$compute_block2 = [scriptblock]::Create($compute_block)
$object = New-Object PSObject `
    | Add-Member ScriptMethod Compute $compute_block2 -PassThru
然后问题就解决了。

有趣的是,在PowerShell v2中,原始脚本(案例2)根本不起作用,它失败了一些奇怪的错误消息。通过修复,它可以在v2和v3中运行。看起来你遇到了一个需要避免的问题。我记不起这样的事了。它可能是一个错误或一个功能。