.NET TPL CancellationToken内存泄漏

时间:2015-05-04 11:52:14

标签: .net vb.net memory-leaks task-parallel-library cancellation-token

我开发了一个实现工作项生产者/消费者模式的库。工作已经出列,并且每个出列的工作项都会出现一个单独的任务,其中包含失败和成功的延续。

任务继续在工作项完成(或失败)后重新排队。

整个库共享一个中心CancellationTokenSource,在应用程序关闭时触发。

我现在面临重大的内存泄漏。如果使用取消令牌作为参数创建任务,则任务似乎保留在内存中,直到触发取消源(并稍后处理)。

这可以在此示例代码(VB.NET)中重现。主要任务是包装工作项的任务,继续任务将处理重新安排。

Dim oCancellationTokenSource As New CancellationTokenSource
Dim oToken As CancellationToken = oCancellationTokenSource.Token
Dim nActiveTasks As Integer = 0

Dim lBaseMemory As Long = GC.GetTotalMemory(True)

For iteration = 0 To 100 ' do this 101 times to see how much the memory increases

  Dim lMemory As Long = GC.GetTotalMemory(True)

  Console.WriteLine("Memory at iteration start: " & lMemory.ToString("N0"))
  Console.WriteLine("  to baseline: " & (lMemory - lBaseMemory).ToString("N0"))

  For i As Integer = 0 To 1000 ' 1001 iterations to get an immediate, measurable impact
    Interlocked.Increment(nActiveTasks)
    Dim outer As Integer = i
    Dim oMainTask As New Task(Sub()
                                ' perform some work
                                Interlocked.Decrement(nActiveTasks)
                              End Sub, oToken)
    Dim inner As Integer = 1
    Dim oFaulted As Task = oMainTask.ContinueWith(Sub()
                                                    Console.WriteLine("Failed " & outer & "." & inner)
                                                    ' if failed, do something with the work and re-queue it, if possible
                                                    ' (imagine code for re-queueing - essentially just a synchronized list.add)

                                                                                                            ' Does not help:
                                                    ' oMainTask.Dispose()
                                                  End Sub, oToken, TaskContinuationOptions.OnlyOnFaulted, TaskScheduler.Default)
    ' if not using token, does not cause increase in memory:
    'End Sub, TaskContinuationOptions.OnlyOnFaulted)

            ' Does not help:
    ' oFaulted.ContinueWith(Sub()
    '                         oFaulted.Dispose()
    '                       End Sub, TaskContinuationOptions.NotOnFaulted)


    Dim oSucceeded As Task = oMainTask.ContinueWith(Sub()
                                                      ' success
                                                      ' re-queue for next iteration
                                                      ' (imagine code for re-queueing - essentially just a synchronized list.add)

                                                                                                                ' Does not help:
                                                      ' oMainTask.Dispose()
                                                    End Sub, oToken, TaskContinuationOptions.OnlyOnRanToCompletion, TaskScheduler.Default)
    ' if not using token, does not cause increase in memory:
    'End Sub, TaskContinuationOptions.OnlyOnRanToCompletion)

            ' Does not help:
    ' oSucceeded.ContinueWith(Sub()
    '                           oSucceeded.Dispose()
    '                         End Sub, TaskContinuationOptions.NotOnFaulted)


    ' This does not help either and makes processing much slower due to the thrown exception (at least one of these tasks is cancelled)
    'Dim oDisposeTask As New Task(Sub()
    '                               Try
    '                                 Task.WaitAll({oMainTask, oFaulted, oSucceeded, oFaultedFaulted, oSuccededFaulted})
    '                               Catch ex As Exception

    '                               End Try
    '                               oMainTask.Dispose()
    '                               oFaulted.Dispose()
    '                               oSucceeded.Dispose()                                     
    '                             End Sub)

    oMainTask.Start()
    '  oDisposeTask.Start()
  Next

  Console.WriteLine("Memory after creating tasks: " & GC.GetTotalMemory(True).ToString("N0"))

  ' Wait until all main tasks are finished (may not mean that continuations finished)

  Dim previousActive As Integer = nActiveTasks
  While nActiveTasks > 0
    If previousActive <> nActiveTasks Then
      Console.WriteLine("Active: " & nActiveTasks)
      Thread.Sleep(500)
      previousActive = nActiveTasks
    End If

  End While

  Console.WriteLine("Memory after tasks finished: " & GC.GetTotalMemory(True).ToString("N0"))

Next

我使用ANTS Memory Profiler测量了内存使用情况,并看到System.Threading.ExecutionContext大幅增加,后者追溯到任务延续和CancellationCallbackInfo

如您所见,我已经尝试处理使用取消令牌的任务,但这似乎没有效果。

修改

我正在使用.NET 4.0

更新

即使只是在主要任务的链接中继续失败,内存使用也会不断增加。任务继续似乎阻止取消令牌注册的取消注册。

因此,如果一个任务链接了一个没有运行的延续(由于TaskContinuationOptions),那么似乎存在内存泄漏。如果只有一个延续运行,那么我没有观察到内存泄漏。

解决方法

作为一种解决方法,我可以在没有任何TaskContinuationOptions的情况下进行单个延续,并在那里处理父任务的状态:

oMainTask.ContinueWith(Sub(t)
                     If t.IsCanceled Then
                       ' ignore
                     ElseIf t.IsCompleted Then
                       ' reschedule

                     ElseIf t.IsFaulted Then
                       ' error handling

                     End If
                   End Sub)

我必须检查在取消的情况下它的表现如何,但这似乎可以解决问题。我几乎怀疑.NET Framework中有一个错误。具有相互排斥条件的任务取消并不是很少见的事情。

2 个答案:

答案 0 :(得分:4)

一些观察

  1. 潜在的泄漏只出现在有任务&#34;分支&#34;不运行。在您的示例中,如果您注释掉oFaulted任务,那么泄漏就会消失。如果您更新代码以导致oMainTask错误,以便oFaulted任务运行且oSucceeded任务未运行,则注释掉oSucceeded可防止泄漏。< / LI>
  2. 也许没有帮助,但如果在所有任务运行后调用oCancellationTokenSource.Cancel(),内存就会释放。处置没有帮助,也没有任何组合处理取消源和任务。
  3. 我看了http://referencesource.microsoft.com/这是4.5.2(有没有办法查看早期的框架?)我知道它不一定相同,但知道发生了什么类型的事情很有用。基本上,当您将取消令牌传递给任务时,该任务会使用取消令牌的取消源注册自己。因此,取消源会保留对您所有任务的引用。我还不清楚为什么你的场景似乎在泄漏。如果我找到任何东西,我有机会深入了解后,我会更新。
  4. 解决方法

    将分支逻辑移动到始终运行的延续。

    Dim continuation As Task =
        oMainTask.ContinueWith(
            Sub(antecendent)
                If antecendent.Status = TaskStatus.Faulted Then
                    'Handle errors
                ElseIf antecendent.Status = TaskStatus.RanToCompletion Then
                    'Do something else
                End If
            End Sub,
            oToken,
            TaskContinuationOptions.None,
            TaskScheduler.Default)
    

    很有可能这比另一种方法更轻。在这两种情况下,总会运行一个延续,但使用此代码只会创建1个连续任务而不是2。

答案 1 :(得分:0)

我能够通过移动这两行来解决.net 4.0下的问题

Dim oCancellationTokenSource As New CancellationTokenSource
Dim oToken As CancellationToken = oCancellationTokenSource.Token

在第一个循环中

然后在那个循环结束时

oToken = Nothing
oCancellationTokenSource.Dispose()

我也移动了

Interlocked.Decrement(nActiveTasks)

每个&#34;最终&#34;

以来的任务
While nActiveTasks > 0

不准确。

这里有工作的代码

Imports System.Threading.Tasks
Imports System.Threading

Module Module1

Sub Main()
    Dim nActiveTasks As Integer = 0

    Dim lBaseMemory As Long = GC.GetTotalMemory(True)

    For iteration = 0 To 100 ' do this 101 times to see how much the memory increases
        Dim oCancellationTokenSource As New CancellationTokenSource
        Dim oToken As CancellationToken = oCancellationTokenSource.Token
        Dim lMemory As Long = GC.GetTotalMemory(True)

        Console.WriteLine("Memory at iteration start: " & lMemory.ToString("N0"))
        Console.WriteLine("  to baseline: " & (lMemory - lBaseMemory).ToString("N0"))

        For i As Integer = 0 To 1000 ' 1001 iterations to get an immediate, measurable impact
            Dim outer As Integer = iteration
            Dim inner As Integer = i

            Interlocked.Increment(nActiveTasks)

            Dim oMainTask As New Task(Sub()
                                          ' perform some work
                                      End Sub, oToken, TaskCreationOptions.None)

            oMainTask.ContinueWith(Sub()
                                       Console.WriteLine("Failed " & outer & "." & inner)
                                       Interlocked.Decrement(nActiveTasks)
                                   End Sub, oToken, TaskContinuationOptions.OnlyOnFaulted, TaskScheduler.Default)


            oMainTask.ContinueWith(Sub()
                                       If inner Mod 250 = 0 Then Console.WriteLine("Success " & outer & "." & inner)
                                       Interlocked.Decrement(nActiveTasks)
                                   End Sub, oToken, TaskContinuationOptions.OnlyOnRanToCompletion, TaskScheduler.Default)


            oMainTask.Start()
        Next

        Console.WriteLine("Memory after creating tasks: " & GC.GetTotalMemory(True).ToString("N0"))


        Dim previousActive As Integer = nActiveTasks
        While nActiveTasks > 0
            If previousActive <> nActiveTasks Then
                Console.WriteLine("Active: " & nActiveTasks)
                Thread.Sleep(500)
                previousActive = nActiveTasks
            End If

        End While

        oToken = Nothing
        oCancellationTokenSource.Dispose()

        Console.WriteLine("Memory after tasks finished: " & GC.GetTotalMemory(True).ToString("N0"))

    Next

    Console.WriteLine("Final Memory after finished: " & GC.GetTotalMemory(True).ToString("N0"))

    Console.Read()
End Sub

End Module