Powershell Runspacepool-为什么无法从同步哈希表输出到.txt文件

时间:2019-12-23 10:43:36

标签: multithreading powershell optimization hashtable runspace

我要做什么

以下脚本循环遍历数据流数组中的每个项目,并请求摘要值以输出到文本文件。到目前为止,此外部请求是该过程中最昂贵的部分,因此我现在使用Runspacepool并行运行多个(5)请求,以先完成结果的输出为准。

这些请求全部写入一个同步哈希表$ hash ,该哈希表保存一个正在运行的总计($ hash.counter)并跟踪哪个线程($ hash.thread)正在更新总计和一个。 txt输出文件,以避免潜在的写冲突。


什么不起作用

每个线程都能足够容易地更新计数器$hash.counter+=$r,但是当我尝试将值读入 Add-Content 语句时: Add-Content C:\Temp\test.txt "$hash.counter|$r|$p|$ThreadID" 它添加了对象引用而不是数字:

  

System.Collections.Hashtable + SyncHashtable.counter | 123 | MyStreamName | 21252

所以我最终将计数器通过一个可以在字符串中使用的临时变量传递:

[int]$t = $hash.counter+0

Add-Content C:\Temp\test.txt "$t|$r|$p|$ThreadID"

哪个会输出真实的总数:

  

14565423 | 123 | MyStreamName | 21252


我在问什么

  1. 是否可以删除此临时变量并直接从哈希表中输出?为什么对象引用的中间有一个“ +”?
  2. 我不得不添加逻辑来“锁定”哈希表,以防止数据冲突。这是否有必要?有人告诉我同步哈希表对于R / W操作应该是线程安全的,但是如果没有这种逻辑,我的计数器就无法达到正确的总数。

下面循环的完整代码-我省略了Runspacepool等的设置

ForEach($i in $Array){
# Save down the data stream name and create parameter list for passing to the new job
$p = $i.Point.Name
$parameters = @{
    hash = $hash
    conn = $Conn
    p = $p
}

# Instantiate new powershell runspace and send a script to it
$PowerShell = [powershell]::Create()
$PowerShell.RunspacePool = $RunspacePool
[void]$Powershell.AddScript({

    # Receive parameter list, retrieve threadid
    Param (
        $hash,
        $conn,
        $p
    )
    $ThreadID = [appdomain]::GetCurrentThreadId()

    # Send data request to the PI Data Archive using the existing connection
    $q = Get-something (actual code removed)
    [int]$r = $q.Values.Values[0].Value

    # Lock out other threads from writing to the hashtable and output file to prevent collisions
    # If the thread isn't locked, attempt to lock it. If it is locked, sleep for 1ms and try again. Tracked by synchronised Hashtable.
    Do{
        if($hash.thread -eq 0){
            $hash.thread = $ThreadID
        }
        # Sleep for 1ms and check that the lock wasn't overwritten by another parallel thread.
        Start-Sleep -Milliseconds 1
    }Until($hash.thread -eq $ThreadID)

    # Increment the synchronised hash counter. Save the new result to a temporary variable (can't figure out how to get the hash counter itself to output to the file)
    $hash.counter+=$r
    [int]$t = $hash.counter+0

    # Write to output file new counter total, result, pointName and threadID
    Add-Content C:\Temp\test.txt "$t|$r|$p|$ThreadID"

    # release lock on the hashtable and output file
    $hash.thread = 0
})

# Add parameter list to instance (matching param() list from the script. Invoke the new instance and save a handle for closing it
[void]$Powershell.AddParameters($parameters)
$Handle = $PowerShell.BeginInvoke()

# Save down the handle into the $jobs list for closing the instances afterwards
$temp = [PSCustomObject]@{
    PowerShell=$Powershell
    Handle=$Handle
} 
[void]$jobs.Add($Temp)
}

0 个答案:

没有答案