Question

如何在Parallel.ForEach循环中控制对共享资源的访问？我正在尝试并行下载多个文件，并且我想捕获有关失败的下载的信息，以便用户可以稍后重新尝试下载。但是，我担心如果多个下载同时失败，应用程序将抛出异常，因为一个线程将在另一个线程写入文件时尝试访问该文件。

在下面的代码中，我想知道如何在RepeateRequestPath上控制对文件的访问。 RequestSet是一个字符串列表，表示我尝试下载的资源的ID。

Dim DownloadCnt As Integer = 0
Dim ParallelOpts As New ParallelOptions()
ParallelOpts.MaxDegreeOfParallelism = 4
Parallel.ForEach(RequestSets, ParallelOpts, Sub(RequestSet)
        Try
            DownloadCnt += 1
            Dim XmlUrl As String = String.Format("{0}{1}{2}", "http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&id=", String.Join(",", RequestSet), "&retmode=xml&rettype=abstract")
            DownloadFile(XmlUrl, String.Format("{0}\TempXML{1}.xml", XMLCacheDir, DownloadCnt))
        Catch ex As WebException
            Using Response As WebResponse = ex.Response
                Dim statCode As Integer = CInt(DirectCast(Response, HttpWebResponse).StatusCode)
                MessageBox.Show(String.Format("Failed to retrieve XML due to HTTP error {0}. Please hit the 'Retrieve XML' button to re-run retrieval after the current set is complete.", statCode))
                If Not File.Exists(RepeatRequestPath) Then
                    File.WriteAllLines(RepeatRequestPath, RequestSet)
                Else
                    File.AppendAllLines(RepeatRequestPath, RequestSet)
                End If
            End Using
        End Try
    End Sub)

Answer 1

在VB.NET中保护共享资源的常用方法是使用SyncLock。

因此，您将在Parallel.ForEach()循环之前创建一个锁定对象：

Dim lock = New Object

然后你会在循环中使用它：

SyncLock lock
    File.AppendAllLines(RepeatRequestPath, RequestSet)
End SyncLock

另请注意，即使文件尚不存在，您也可以使用AppendAllLines()，因此您无需检查该文件。

Answer 2

您需要使用信号量来控制对共享资源的访问。您只需要一个线程一次访问错误文件，因此初始化信号量只允许1个线程进入。调用_pool.WaitOne应该获取信号量，然后在完成创建/写入文件后释放它。

Private Shared _pool As Semaphore
_pool = = New Semaphore(0, 1)

Dim DownloadCnt As Integer = 0
Dim ParallelOpts As New ParallelOptions()
ParallelOpts.MaxDegreeOfParallelism = 4
Parallel.ForEach(RequestSets, ParallelOpts, Sub(RequestSet)
        Try
            DownloadCnt += 1
            Dim XmlUrl As String = String.Format("{0}{1}{2}", "http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&id=", String.Join(",", RequestSet), "&retmode=xml&rettype=abstract")
            DownloadFile(XmlUrl, String.Format("{0}\TempXML{1}.xml", XMLCacheDir, DownloadCnt))
        Catch ex As WebException
            Using Response As WebResponse = ex.Response
                Dim statCode As Integer = CInt(DirectCast(Response, HttpWebResponse).StatusCode)
                MessageBox.Show(String.Format("Failed to retrieve XML due to HTTP error {0}. Please hit the 'Retrieve XML' button to re-run retrieval after the current set is complete.", statCode))
                _pool.WaitOne()
                Try
                    If Not File.Exists(RepeatRequestPath) Then
                        File.WriteAllLines(RepeatRequestPath, RequestSet)
                    Else
                        File.AppendAllLines(RepeatRequestPath, RequestSet)
                    End If
                Catch ex as Exception
                    'Do some error handling here.
                Finally
                    _pool.Release()
                End Try
            End Using
        End Try
    End Sub)

Answer 3

svick的解决方案几乎是正确的。但是，如果您需要保护对共享变量的访问权限，则还需要将您的锁定对象声明为在类级别共享。

这是正常的：

Friend Class SomeClass
    Private Shared _lock As New Object

    Private Shared sharedInt As Integer = 0

    Sub Main()
        SyncLock _lock
            sharedInt += 1
        End SyncLock
    End Sub
End Class

如果使用非共享锁对象，则synclock将仅保护变量，使其仅保护同一实例中的多个访问线程，而不是跨实例。

Parallel.ForEach中的共享资源

3 个答案: