我开发了一个实现工作项生产者/消费者模式的库。工作已经出列,并且每个出列的工作项都会出现一个单独的任务,其中包含失败和成功的延续。
任务继续在工作项完成(或失败)后重新排队。
整个库共享一个中心CancellationTokenSource
,在应用程序关闭时触发。
我现在面临重大的内存泄漏。如果使用取消令牌作为参数创建任务,则任务似乎保留在内存中,直到触发取消源(并稍后处理)。
这可以在此示例代码(VB.NET)中重现。主要任务是包装工作项的任务,继续任务将处理重新安排。
Dim oCancellationTokenSource As New CancellationTokenSource
Dim oToken As CancellationToken = oCancellationTokenSource.Token
Dim nActiveTasks As Integer = 0
Dim lBaseMemory As Long = GC.GetTotalMemory(True)
For iteration = 0 To 100 ' do this 101 times to see how much the memory increases
Dim lMemory As Long = GC.GetTotalMemory(True)
Console.WriteLine("Memory at iteration start: " & lMemory.ToString("N0"))
Console.WriteLine(" to baseline: " & (lMemory - lBaseMemory).ToString("N0"))
For i As Integer = 0 To 1000 ' 1001 iterations to get an immediate, measurable impact
Interlocked.Increment(nActiveTasks)
Dim outer As Integer = i
Dim oMainTask As New Task(Sub()
' perform some work
Interlocked.Decrement(nActiveTasks)
End Sub, oToken)
Dim inner As Integer = 1
Dim oFaulted As Task = oMainTask.ContinueWith(Sub()
Console.WriteLine("Failed " & outer & "." & inner)
' if failed, do something with the work and re-queue it, if possible
' (imagine code for re-queueing - essentially just a synchronized list.add)
' Does not help:
' oMainTask.Dispose()
End Sub, oToken, TaskContinuationOptions.OnlyOnFaulted, TaskScheduler.Default)
' if not using token, does not cause increase in memory:
'End Sub, TaskContinuationOptions.OnlyOnFaulted)
' Does not help:
' oFaulted.ContinueWith(Sub()
' oFaulted.Dispose()
' End Sub, TaskContinuationOptions.NotOnFaulted)
Dim oSucceeded As Task = oMainTask.ContinueWith(Sub()
' success
' re-queue for next iteration
' (imagine code for re-queueing - essentially just a synchronized list.add)
' Does not help:
' oMainTask.Dispose()
End Sub, oToken, TaskContinuationOptions.OnlyOnRanToCompletion, TaskScheduler.Default)
' if not using token, does not cause increase in memory:
'End Sub, TaskContinuationOptions.OnlyOnRanToCompletion)
' Does not help:
' oSucceeded.ContinueWith(Sub()
' oSucceeded.Dispose()
' End Sub, TaskContinuationOptions.NotOnFaulted)
' This does not help either and makes processing much slower due to the thrown exception (at least one of these tasks is cancelled)
'Dim oDisposeTask As New Task(Sub()
' Try
' Task.WaitAll({oMainTask, oFaulted, oSucceeded, oFaultedFaulted, oSuccededFaulted})
' Catch ex As Exception
' End Try
' oMainTask.Dispose()
' oFaulted.Dispose()
' oSucceeded.Dispose()
' End Sub)
oMainTask.Start()
' oDisposeTask.Start()
Next
Console.WriteLine("Memory after creating tasks: " & GC.GetTotalMemory(True).ToString("N0"))
' Wait until all main tasks are finished (may not mean that continuations finished)
Dim previousActive As Integer = nActiveTasks
While nActiveTasks > 0
If previousActive <> nActiveTasks Then
Console.WriteLine("Active: " & nActiveTasks)
Thread.Sleep(500)
previousActive = nActiveTasks
End If
End While
Console.WriteLine("Memory after tasks finished: " & GC.GetTotalMemory(True).ToString("N0"))
Next
我使用ANTS Memory Profiler测量了内存使用情况,并看到System.Threading.ExecutionContext大幅增加,后者追溯到任务延续和CancellationCallbackInfo
。
如您所见,我已经尝试处理使用取消令牌的任务,但这似乎没有效果。
修改
我正在使用.NET 4.0
更新
即使只是在主要任务的链接中继续失败,内存使用也会不断增加。任务继续似乎阻止取消令牌注册的取消注册。
因此,如果一个任务链接了一个没有运行的延续(由于TaskContinuationOptions
),那么似乎存在内存泄漏。如果只有一个延续运行,那么我没有观察到内存泄漏。
解决方法
作为一种解决方法,我可以在没有任何TaskContinuationOptions
的情况下进行单个延续,并在那里处理父任务的状态:
oMainTask.ContinueWith(Sub(t)
If t.IsCanceled Then
' ignore
ElseIf t.IsCompleted Then
' reschedule
ElseIf t.IsFaulted Then
' error handling
End If
End Sub)
我必须检查在取消的情况下它的表现如何,但这似乎可以解决问题。我几乎怀疑.NET Framework中有一个错误。具有相互排斥条件的任务取消并不是很少见的事情。
答案 0 :(得分:4)
一些观察
oFaulted
任务,那么泄漏就会消失。如果您更新代码以导致oMainTask
错误,以便oFaulted
任务运行且oSucceeded
任务未运行,则注释掉oSucceeded
可防止泄漏。< / LI>
oCancellationTokenSource.Cancel()
,内存就会释放。处置没有帮助,也没有任何组合处理取消源和任务。解决方法强>
将分支逻辑移动到始终运行的延续。
Dim continuation As Task =
oMainTask.ContinueWith(
Sub(antecendent)
If antecendent.Status = TaskStatus.Faulted Then
'Handle errors
ElseIf antecendent.Status = TaskStatus.RanToCompletion Then
'Do something else
End If
End Sub,
oToken,
TaskContinuationOptions.None,
TaskScheduler.Default)
很有可能这比另一种方法更轻。在这两种情况下,总会运行一个延续,但使用此代码只会创建1个连续任务而不是2。
答案 1 :(得分:0)
我能够通过移动这两行来解决.net 4.0下的问题
Dim oCancellationTokenSource As New CancellationTokenSource
Dim oToken As CancellationToken = oCancellationTokenSource.Token
在第一个循环中
然后在那个循环结束时
oToken = Nothing
oCancellationTokenSource.Dispose()
我也移动了
Interlocked.Decrement(nActiveTasks)
每个&#34;最终&#34;
以来的任务While nActiveTasks > 0
不准确。
这里有工作的代码
Imports System.Threading.Tasks
Imports System.Threading
Module Module1
Sub Main()
Dim nActiveTasks As Integer = 0
Dim lBaseMemory As Long = GC.GetTotalMemory(True)
For iteration = 0 To 100 ' do this 101 times to see how much the memory increases
Dim oCancellationTokenSource As New CancellationTokenSource
Dim oToken As CancellationToken = oCancellationTokenSource.Token
Dim lMemory As Long = GC.GetTotalMemory(True)
Console.WriteLine("Memory at iteration start: " & lMemory.ToString("N0"))
Console.WriteLine(" to baseline: " & (lMemory - lBaseMemory).ToString("N0"))
For i As Integer = 0 To 1000 ' 1001 iterations to get an immediate, measurable impact
Dim outer As Integer = iteration
Dim inner As Integer = i
Interlocked.Increment(nActiveTasks)
Dim oMainTask As New Task(Sub()
' perform some work
End Sub, oToken, TaskCreationOptions.None)
oMainTask.ContinueWith(Sub()
Console.WriteLine("Failed " & outer & "." & inner)
Interlocked.Decrement(nActiveTasks)
End Sub, oToken, TaskContinuationOptions.OnlyOnFaulted, TaskScheduler.Default)
oMainTask.ContinueWith(Sub()
If inner Mod 250 = 0 Then Console.WriteLine("Success " & outer & "." & inner)
Interlocked.Decrement(nActiveTasks)
End Sub, oToken, TaskContinuationOptions.OnlyOnRanToCompletion, TaskScheduler.Default)
oMainTask.Start()
Next
Console.WriteLine("Memory after creating tasks: " & GC.GetTotalMemory(True).ToString("N0"))
Dim previousActive As Integer = nActiveTasks
While nActiveTasks > 0
If previousActive <> nActiveTasks Then
Console.WriteLine("Active: " & nActiveTasks)
Thread.Sleep(500)
previousActive = nActiveTasks
End If
End While
oToken = Nothing
oCancellationTokenSource.Dispose()
Console.WriteLine("Memory after tasks finished: " & GC.GetTotalMemory(True).ToString("N0"))
Next
Console.WriteLine("Final Memory after finished: " & GC.GetTotalMemory(True).ToString("N0"))
Console.Read()
End Sub
End Module