我试图找出我们在服务器应用程序中遇到System.OutOfMemoryException的原因。这台机器有256Gb内存和48个内核,当它使用大约50Gb - 70Gb时会出现例外情况,因此会有足够的空间。
我有一个转储,但不幸的是只有在堆栈被解除之后。使用WinDbg和这个命令:
.foreach(tempVariable {!dumpheap -type System.OutOfMemoryException -short}){!pe -nested tempVariable;.echo *************}
我可以看到这种类型的堆上存在很多异常:
Exception object: 0000009034f610d0
Exception type: System.OutOfMemoryException
Message: <none>
InnerException: <none>
StackTrace (generated):
<none>
StackTraceString: <none>
HResult: 8007000e
Nested exception -------------------------------------------------------------
Exception object: 00000097711dca00
Exception type: System.OutOfMemoryException
Message: <none>
InnerException: <none>
StackTrace (generated):
SP IP Function
0000009FF5115F70 0000000000000000 mscorlib_ni!System.Environment.GetResourceFromDefault(System.String)+0x1
0000009FF5115F70 00007FF83630240E mscorlib_ni!System.Environment.GetResourceString(System.String, System.Object[])+0xe
0000009FF5115FB0 00007FF7DA46CCAC UNKNOWN!common2.taskservice.SimpleLockingTaskQueue.RunReadonlyEvent(TaskQueueEntry)+0xfbc
0000009FF511E7E0 00007FF7DA46BCD1 UNKNOWN!common2.taskservice.SimpleLockingTaskQueue+<>c__DisplayClass31_0.<MainQueueWorkerThread>b__1()+0x31
0000009FF511E810 00007FF83638D436 mscorlib_ni!System.Threading.Tasks.Task.Execute()+0x46
StackTraceString: <none>
HResult: 8007000e
*************
Exception object: 0000009056032ef0
Exception type: System.OutOfMemoryException
Message: <none>
InnerException: <none>
StackTrace (generated):
SP IP Function
0000009F533FE430 0000000000000000 mscorlib_ni!System.Threading.Monitor.Enter(System.Object)+0x1
0000009F533FE430 00007FF7DA361FD0 UNKNOWN!common2.objectevents.RequestEventHandler`3[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].EventCall(System.Action)+0xa0
0000009F533FE7C0 00007FF7DA609774 UNKNOWN!common2.objectevents.RequestEventHandler`3[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].<ExecuteTail>b__56_0()+0x44
0000009F533FE7F0 00007FF7DA46BEBB UNKNOWN!common2.taskservice.SimpleLockingTaskQueue.RunReadonlyEvent(TaskQueueEntry)+0x1cb
因此几乎所有的OutOfMemoryException都在GetResourceFromDefault或Monitor.Enter()中。我想第一个处理程序试图分配内存来显示异常的语言环境版本?但是Monitor.Enter()呢?
UNKNOWN模块当然是我们的应用程序..
任何人都有任何线索可以找出导致异常的确切原因?
修改:更多信息
!gchandles
...
Statistics:
MT Count TotalSize Class Name
00007ff83993f3b0 1 40 System.Xml.Linq.XNamespace
00007ff836567040 1 48 System.SharedStatics
00007ff835e7b208 1 64 System.EventHandler`1[[Windows.Foundation.Diagnostics.TracingStatusChangedEventArgs, mscorlib]]
00007ff836566f28 3 72 System.Object
00007ff836534af8 1 80 System.Threading.PinnableBufferCache
00007ff831f51b88 1 80 System.Diagnostics.BooleanSwitch
00007ff831f45468 3 120 System.Net.TimerThread+TimerQueue
00007ff836567f70 2 128 System.Security.PermissionSet
00007ff836566e50 1 160 System.ExecutionEngineException
00007ff836566dd8 1 160 System.StackOverflowException
00007ff836566b78 1 160 System.Exception
00007ff836563900 1 192 System.Diagnostics.Tracing.FrameworkEventSource
00007ff836559a50 1 192 System.Threading.Tasks.TplEtwProvider
00007ff836548898 1 192 System.Collections.Concurrent.CDSCollectionETWBCLProvider
00007ff836534b58 1 192 System.Threading.PinnableBufferCacheEventSource
00007ff836525928 1 192 System.Threading.CdsSyncEtwBCLProvider
00007ff831f356e8 1 192 System.PinnableBufferCacheEventSource
00007ff82d10a570 1 192 System.Web.TelemetryEventSource
00007ff836567100 1 216 System.AppDomain
00007ff836574be0 2 320 System.NotSupportedException
00007ff836566ec8 2 320 System.Threading.ThreadAbortException
00007ff831f39f88 4 320 System.PinnableBufferCache
00007ff831f45658 6 336 System.Net.Logging+NclTraceSource
00007ff8365602b0 7 448 Microsoft.Win32.UnsafeNativeMethods+ManifestEtw+EtwEnableCallback
00007ff831f45740 6 480 System.Diagnostics.SourceSwitch
00007ff831f34d10 3 600 System.Net.ServicePoint
00007ff7d98575e0 4 704 common2.objectevents.GlobalCacheRetryException
00007ff8388c0ca8 33 2112 System.DirectoryServices.Protocols.VERIFYSERVERCERT
00007ff8388c5150 33 3168 System.DirectoryServices.Protocols.LdapConnection
00007ff83656c988 7 20504 System.Byte[]
00007ff836566d60 153 24480 System.OutOfMemoryException
00007ff836567d28 1263 121248 System.Threading.Thread
00007ff836548510 1 156336 System.Int64[]
00007ff83654a7d0 3835 276120 System.Reflection.Emit.DynamicResolver
00007ff83657fc70 2084 333440 System.RuntimeType+RuntimeTypeCache
00007ff836566fc0 480 502616 System.Object[]
00007ff835e6f918 90017 10081904 System.Threading.OverlappedData
Total 97964 objects
Handles:
Strong Handles: 815
Pinned Handles: 23
Async Pinned Handles: 90017
Ref Count Handles: 1
Weak Long Handles: 5959
Weak Short Handles: 1149
错误线程和堆栈
0:134> ~#s
ntdll!NtWaitForSingleObject+0xa:
00007ff8`4b19070a c3 ret
0:134> !pe
Exception object: 00000097711df1a8
Exception type: System.OutOfMemoryException
Message: <none>
InnerException: <none>
StackTrace (generated):
SP IP Function
0000009FF5110C60 0000000000000000 mscorlib_ni!System.Threading.Monitor.Enter(System.Object)+0x1
0000009FF5110C60 00007FF83638C87E mscorlib_ni!System.Threading.Tasks.Task.AddException(System.Object, Boolean)+0xae
0000009FF5110CD0 00007FF836D1B4C8 mscorlib_ni!System.Threading.Tasks.Task.HandleException(System.Exception)+0x88
0000009FF5110D20 00007FF83638D4AB mscorlib_ni!System.Threading.Tasks.Task.Execute()+0xbb
0000009FF511E850 00007FF83633CA72 mscorlib_ni!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)+0x162
0000009FF511E920 00007FF83633C904 mscorlib_ni!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)+0x14
0000009FF511E950 00007FF83638D6DC mscorlib_ni!System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef)+0x21c
0000009FF511EA00 00007FF83638CDF3 mscorlib_ni!System.Threading.Tasks.Task.ExecuteEntry(Boolean)+0x73
0000009FF511EA40 00007FF836374882 mscorlib_ni!System.Threading.ThreadPoolWorkQueue.Dispatch()+0x152
免费物品堆
0:134> !dumpheap -stat -type Free
Statistics:
MT Count TotalSize Class Name
00007ff831f35af8 1 32 System.Net.SafeLocalFree
00007ff831f39188 1 48 System.Net.SafeFreeCredential_SECURITY
00007ff831f3abd0 11 440 System.Net.SafeFreeContextBufferChannelBinding_SECURITY
0000009032e03d20 18266055 5247426202 Free
Total 18266068 objects
Fragmented blocks larger than 0.5 MB:
Addr Size Followed by
0000009052cca810 13.7MB 0000009053a72bc0 dbtaskmessages.KeyValueObject
00000090ef377af0 11.4MB 00000090efedffe0 System.String
00000090f14cf6c0 20.8MB 00000090f2996d18 dbtaskmessages.KeyValueObject
000000912a19b2b8 12.6MB 000000912ae3dbb0 System.Collections.Concurrent.ConcurrentDictionary`2+Node[[System.Int64, mscorlib],[System.Object, mscorlib]]
00000091ec96bbd0 10.3MB 00000091ed3bd0f8 System.Collections.Concurrent.ConcurrentDictionary`2+Node[[System.Int64, mscorlib],[System.Object, mscorlib]]
00000091ed8e66f8 13.0MB 00000091ee5efe20 System.String
00000091efce52b8 10.1MB 00000091f07067b0 System.Collections.Concurrent.ConcurrentDictionary`2+Node[[System.Int64, mscorlib],[System.Object, mscorlib]]
000000922a3946e0 10.1MB 000000922ada5c48 System.Collections.Concurrent.ConcurrentQueue`1+Segment[[System.Action, mscorlib]]
000000926c6395f8 15.6MB 000000926d5d02e0 System.Int32[]
000000943169ca50 11.4MB 000000943220d3c0 System.Collections.Concurrent.ConcurrentDictionary`2+Node[[System.Int64, mscorlib],[System.Object, mscorlib]]
00000094a39033d0 12.4MB 00000094a4573ee0 System.Collections.Concurrent.ConcurrentDictionary`2+Node[[System.Int64, mscorlib],[System.Object, mscorlib]]
000000955e5ef658 10.3MB 000000955f044318 common2.database.GlobalCaching.CachedObjectQueryTable`3+CachedCollection[[HnG_States.ProtoObjects.SocialConnection, HnG_States],[HnG_States.ProtoObjects.Indexes.SocialConnectionIndex+IndexByInviterAndInvitee, HnG_States],[System.String, mscorlib]]
00000098dcae2108 10.6MB 00000098dd57b508 common2.database.CachingKeyValueObject
00000098e7f6cb68 14.2MB 00000098e8d9da30 System.String
0000009ba8be9868 10.4MB 0000009ba9655030 Microsoft.Win32.SafeHandles.SafeWaitHandle
0000009baf6351a0 23.9MB 0000009bb0e18a20 System.UInt64[]
0000009be4ae9160 12.8MB 0000009be57a94c8 System.String
此列表仅包含对象&gt; 10Mb - 总共有1145个对象
来自错误线程的CLR堆栈
0:134> !clrstack
OS Thread Id: 0x1778 (134)
Child SP IP Call Site
0000009ff5110b68 00007ff84b19070a [HelperMethodFrame_1OBJ: 0000009ff5110b68] System.Threading.Monitor.Enter(System.Object)
0000009ff5110c60 00007ff83638c87e System.Threading.Tasks.Task.AddException(System.Object, Boolean) [f:\dd\ndp\clr\src\BCL\system\threading\Tasks\Task.cs @ 2041]
0000009ff5110cd0 00007ff836d1b4c8 System.Threading.Tasks.Task.HandleException(System.Exception) [f:\dd\ndp\clr\src\BCL\system\threading\Tasks\Task.cs @ 2929]
0000009ff5110d20 00007ff83638d4ab System.Threading.Tasks.Task.Execute() [f:\dd\ndp\clr\src\BCL\system\threading\Tasks\Task.cs @ 2515]
0000009ff5115cd8 00007ff838d3120d [GCFrame: 0000009ff5115cd8]
0000009ff5115e78 00007ff838d3120d [HelperMethodFrame_2OBJ: 0000009ff5115e78] System.Environment.GetResourceFromDefault(System.String)
0000009ff5115f70 00007ff83630240e System.Environment.GetResourceString(System.String, System.Object[]) [f:\dd\ndp\clr\src\BCL\system\environment.cs @ 1332]
0000009ff5115fb0 00007ff7da46ccac common2.taskservice.SimpleLockingTaskQueue.RunReadonlyEvent(TaskQueueEntry)
0000009ff511af78 00007ff838d3120d [GCFrame: 0000009ff511af78]
0000009ff511b118 00007ff838d3120d [HelperMethodFrame_2OBJ: 0000009ff511b118] System.Environment.GetResourceFromDefault(System.String)
0000009ff511b210 00007ff83630240e System.Environment.GetResourceString(System.String, System.Object[]) [f:\dd\ndp\clr\src\BCL\system\environment.cs @ 1332]
0000009ff511b250 00007ff836386e87 System.Exception.ToString(Boolean, Boolean) [f:\dd\ndp\clr\src\BCL\system\exception.cs @ 439]
0000009ff511b2a0 00007ff7da362b87 common2.objectevents.RequestEventHandler`3[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].EventCall(System.Action)
0000009ff511d408 00007ff838d3120d [HelperMethodFrame_1OBJ: 0000009ff511d408] System.Threading.Monitor.Enter(System.Object)
0000009ff511d500 00007ff7da0aa16a System.Collections.Concurrent.ConcurrentDictionary`2[[System.Int64, mscorlib],[System.__Canon, mscorlib]].AcquireLocks(Int32, Int32, Int32 ByRef) [f:\dd\ndp\clr\src\BCL\system\Collections\Concurrent\ConcurrentDictionary.cs @ 1911]
0000009ff511d560 00007ff7da489e91 System.Collections.Concurrent.ConcurrentDictionary`2[[System.Int64, mscorlib],[System.__Canon, mscorlib]].GetKeys() [f:\dd\ndp\clr\src\BCL\system\Collections\Concurrent\ConcurrentDictionary.cs @ 1955]
0000009ff511d5c0 00007ff7da489c4a common2.database.KeyIdTable`1[[System.__Canon, mscorlib]].GetIds(System.Collections.Generic.List`1, System.__Canon)
0000009ff511d620 00007ff7da489864 common2.database.SimpleObjectQueryTable`3[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].Get(System.__Canon, common2.database.RetrieveContext)
0000009ff511d7d0 00007ff7da4891e5 common2.database.ObjectTable.Query[[System.__Canon, mscorlib]](System.__Canon, common2.database.RetrieveContext)
0000009ff511d880 00007ff7da488ecf common2.database.GlobalCaching.ContextualPureObjectQuery`1[[System.__Canon, mscorlib]].Query[[System.__Canon, mscorlib]](System.__Canon)
0000009ff511d980 00007ff7da488236 common2.database.CacheObjectQuery`1[[System.__Canon, mscorlib]].Query[[System.__Canon, mscorlib]](System.__Canon)
0000009ff511daa0 00007ff7da5d661d HnG_States.timeevent_extensions.By_ObjType_ObjId(common2.database.Interfaces.IObjectTableQuery`1, System.String, Int64)
0000009ff511db00 00007ff7da5cb610 WarServer.StrategyCleanup_CronJob.CleanupTransports(common2.database.Interfaces.IObjectDbConnection2, HnG_States.ProtoObjects.war)
0000009ff511dca0 00007ff7da5c9aab WarServer.StrategyCleanup_CronJob.HandleCronJob()
0000009ff511dd70 00007ff7da5c28fa common2.objectevents.CronJobRequestEventHandler`3[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].Handle()
0000009ff511deb0 00007ff7da3623ec common2.objectevents.RequestEventHandler`3[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].EventCall(System.Action)
0000009ff511e240 00007ff7da361eaf common2.objectevents.RequestEventHandler`3[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].HandleEvent()
0000009ff511e2e0 00007ff7da5ae7a0 common2.objectevents.ClientRequestEventHandler`3[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].HandleEvent()
0000009ff511e390 00007ff7da46bebb common2.taskservice.SimpleLockingTaskQueue.RunReadonlyEvent(TaskQueueEntry)
0000009ff511e7e0 00007ff7da46bcd1 common2.taskservice.SimpleLockingTaskQueue+c__DisplayClass31_0.b__1()
0000009ff511e810 00007ff83638d436 System.Threading.Tasks.Task.Execute() [f:\dd\ndp\clr\src\BCL\system\threading\Tasks\Task.cs @ 2498]
0000009ff511e850 00007ff83633ca72 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean) [f:\dd\ndp\clr\src\BCL\system\threading\executioncontext.cs @ 954]
0000009ff511e920 00007ff83633c904 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean) [f:\dd\ndp\clr\src\BCL\system\threading\executioncontext.cs @ 902]
0000009ff511e950 00007ff83638d6dc System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef) [f:\dd\ndp\clr\src\BCL\system\threading\Tasks\Task.cs @ 2827]
0000009ff511ea00 00007ff83638cdf3 System.Threading.Tasks.Task.ExecuteEntry(Boolean) [f:\dd\ndp\clr\src\BCL\system\threading\Tasks\Task.cs @ 2767]
0000009ff511ea40 00007ff836374882 System.Threading.ThreadPoolWorkQueue.Dispatch() [f:\dd\ndp\clr\src\BCL\system\threading\threadpool.cs @ 820]
0000009ff511eed8 00007ff838bd6793 [DebuggerU2MCatchHandlerFrame: 0000009ff511eed8]
来自!threads的压缩输出
0:134> !threads
ThreadCount: 1154
UnstartedThread: 0
BackgroundThread: 243
PendingThread: 0
DeadThread: 905
Hosted Runtime: no
Lock
ID OSID ThreadOBJ State GC Mode GC Alloc Context Domain Count Apt Exception
134 1111 1778 0000009f5fcc1e00 10a9228 Preemptive 00000097711E0D00:00000097711E1190 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 00000097711df1a8 (nested exceptions)
135 1250 1288 0000009fef1bc510 1029228 Preemptive 000000A4D5D27438:000000A4D5D28868 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000a4d5d258e0 (nested exceptions)
136 1298 c20 0000009f5fc995c0 10a9228 Preemptive 0000009A2E09FA40:0000009A2E0A02A0 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 0000009a2e098cd0 (nested exceptions)
138 1332 34c8 0000009f5e739410 1029228 Preemptive 000000A4447D6088:000000A4447D68E8 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000a4447d1198 (nested exceptions)
137 1309 1bac 0000009f5fd1ded0 1029228 Preemptive 000000905604D918:000000905604E178 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 0000009056033c68 (nested exceptions)
139 1064 2378 000000a5ed90a830 1029228 Preemptive 000000A3063531D0:000000A306353A30 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000a30634e778 (nested exceptions)
141 1232 33a8 0000009f5ead9840 1029228 Preemptive 000000A87D726CD0:000000A87D727DC0 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000a87d725178 (nested exceptions)
142 1136 1fac 000000900689acc0 1029228 Preemptive 00000099EE7269F0:00000099EE726B30 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 00000099ee724e98 (nested exceptions)
143 1377 3540 000000a045163fd0 10a9228 Preemptive 00000095667C19F8:00000095667C29A8 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 00000095667bfea0 (nested exceptions)
144 1241 d00 000000a010ab3870 1029228 Preemptive 0000009165F29128:0000009165F29988 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 0000009165f229c8 (nested exceptions)
145 1242 2704 00000090071ff070 10a9228 Preemptive 000000952E0F5840:000000952E0F60A0 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000952baa2ec8 (nested exceptions)
147 674 2f2c 0000009f5dfcb720 10a9228 Preemptive 000000A1DB23A7F8:000000A1DB23C388 0000009032df1aa0 0 MTA (Threadpool Worker) common2.objectevents.GlobalCacheRetryException 000000a1db23a6c8
151 1226 3584 000000900447a040 1029228 Preemptive 00000099308F2770:00000099308F2FD0 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000993089daa8 (nested exceptions)
152 1090 19cc 0000009f5fb2ba50 1029228 Preemptive 000000A306345418:000000A3063469C0 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000a3063437e0 (nested exceptions)
155 1209 34ec 0000009f5e96df00 1029228 Preemptive 000000A1DA486760:000000A1DA486FC0 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000a1da40bdd8 (nested exceptions)
154 1163 1dac 00000090064c7010 1029228 Preemptive 000000905604B918:000000905604C178 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000a6bd558138 (nested exceptions)
156 1187 1ea8 0000009006e05630 10a9228 Preemptive 000000A718498E00:000000A7184993F0 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000a71848f1f0 (nested exceptions)
157 248 2a9c 000000a82f251800 10a9228 Preemptive 000000A5F3B38500:000000A5F3B39398 0000009032df1aa0 1 MTA (Threadpool Worker) System.NotSupportedException 000000a5f3b366f0
158 1339 21ec 000000a036221870 1029228 Preemptive 0000009C2A8F2D38:0000009C2A8F3400 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 0000009c2a8bf188 (nested exceptions)
159 1171 194 0000009fef2b0dd0 1029228 Preemptive 000000A5F383A1B0:000000A5F383B368 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000a5f3838658 (nested exceptions)
161 1215 2224 000000a5ed90c770 1029228 Preemptive 000000A22583CDB0:000000A22583D610 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000a225035148 (nested exceptions)
162 1073 1e68 0000009fef17d350 10a9228 Preemptive 000000A8360DD5D0:000000A8360DDB90 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000a8360dba78 (nested exceptions)
163 1381 34a4 0000009004477160 1029228 Preemptive 000000A718488710:000000A7184893F0 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000a718486bb8 (nested exceptions)
164 1365 3684 0000009006b62e90 10a9228 Preemptive 0000009B2EAC16D8:0000009B2EAC3648 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 0000009b2eab7b48 (nested exceptions)
171 1323 13f0 0000009f413dc5f0 10a9228 Preemptive 000000A87DF9FB48:000000A87DF9FC88 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000a87df9dff0 (nested exceptions)
175 1274 1368 000000a300c6ffc0 10a9228 Preemptive 0000009C2A9D0C00:0000009C2A9D1460 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 0000009c2a9bc720 (nested exceptions)
166 1240 34b4 0000009fef17eac0 1029228 Preemptive 000000A2C1FEAE78:000000A2C1FEB6D8 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000a2c1fda230 (nested exceptions)
174 1251 3014 000000a13449a060 1029228 Preemptive 000000952DC45020:000000952DC46B68 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000952dc434c8 (nested exceptions)
168 1362 147c 000000a01d889030 10a9228 Preemptive 000000A6BE3C7F18:000000A6BE3C8778 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000a6be3b96e0 (nested exceptions)
167 1366 23d4 0000009004c7c830 1029228 Preemptive 00000096AC78B9E0:00000096AC78C240 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 00000096ac6b88d0 (nested exceptions)
179 1324 928 0000009f41579010 1029228 Preemptive 00000099AD7D8D48:00000099AD7DAD38 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 00000099ad7d71f0 (nested exceptions)
177 1221 13e4 000000a045164f70 1029228 Preemptive 000000A5A1495748:000000A5A1495FA8 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000a5a145ac98 (nested exceptions)
178 1293 23a8 0000009006bf9af0 1029228 Preemptive 00000094AE08CFE0:00000094AE08EF58 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 00000094ae08b488 (nested exceptions)
181 1349 29e4 000000a045162090 1029228 Preemptive 0000009165F272C0:0000009165F27988 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 0000009165f1f6a8 (nested exceptions)
184 1296 1f84 0000009f417a4dc0 1029228 Preemptive 0000009A2D8D3A98:0000009A2D8D42F8 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 0000009a2d84dcd8 (nested exceptions)
193 1102 11a4 0000009006a26610 1029228 Preemptive 000000A2764CC658:000000A2764CDBD8 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000a2764cab00 (nested exceptions)
186 1210 25ec 0000009f5eadbf50 1029228 Preemptive 00000096AC6B76E0:00000096AC6B79A8 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 00000096ac6b5b88 (nested exceptions)
188 1379 13a4 000000a3678a7e20 1029228 Preemptive 000000A36F372090:000000A36F3728F0 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000a36f3059d8 (nested exceptions)
190 1255 17d0 000000a3678a9590 1029228 Preemptive 000000986D851380:000000986D853138 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000986d84f828 (nested exceptions)
191 1235 193c 0000009f4769c770 1029228 Preemptive 000000A834B596F0:000000A834B59F50 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 000000a834ad32f0 (nested exceptions)
205 783 2fcc 0000009f46811e50 10a9228 Preemptive 0000009BAF601AC8:0000009BAF603188 0000009032df1aa0 0 MTA (Threadpool Worker) common2.objectevents.GlobalCacheRetryException 0000009baf6016b0
208 1357 3ac4 0000009f5dfcaf50 1029228 Preemptive 0000009165F25128:0000009165F25988 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 0000009165f1c3b8 (nested exceptions)
210 1243 2884 0000009fef2af660 1029228 Preemptive 00000099EE7122D0:00000099EE712B30 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 00000099ee6ff608 (nested exceptions)
211 1239 263c 000000a810bcde10 1029228 Preemptive 0000009464764D30:0000009464764F30 0000009032df1aa0 0 MTA (Threadpool Worker) System.OutOfMemoryException 00000094647631d8 (nested exceptions)
219 1183 3820 000000a34ad75a10 8029228 Preemptive 000000986D8548D8:000000986D855138 0000009032df1aa0 0 MTA (Threadpool Completion Port) System.OutOfMemoryException 000000986d83e5f0 (nested exceptions)
220 1217 1968 0000009fef12abb0 8029228 Preemptive 00000099EE716110:00000099EE716B30 0000009032df1aa0 0 MTA (Threadpool Completion Port) System.OutOfMemoryException 00000099ee713f70 (nested exceptions)
224 1335 1e44 0000009f5dfc9010 8029228 Preemptive 000000A87D7BD2F0:000000A87D7BDE50 0000009032df1aa0 0 MTA (Threadpool Completion Port) System.OutOfMemoryException 000000a87d7bb150 (nested exceptions)
223 1327 3244 0000009f418eb490 8029228 Preemptive 000000A5A149AD40:000000A5A149BFA8 0000009032df1aa0 0 MTA (Threadpool Completion Port) System.OutOfMemoryException 000000a5a1498ba0 (nested exceptions)
225 1228 4e0 000000a01e879e10 8029228 Preemptive 000000A3FDE5D5C0:000000A3FDE5DE20 0000009032df1aa0 0 MTA (Threadpool Completion Port) System.OutOfMemoryException 000000a3fde548e0 (nested exceptions)
229 1354 3550 0000009006873d60 8029228 Preemptive 00000094EEB90F28:00000094EEB91788 0000009032df1aa0 0 MTA (Threadpool Completion Port) System.OutOfMemoryException 00000094eeb8c548 (nested exceptions)
230 1152 2b3c 0000009006875ca0 8029228 Preemptive 0000009AE81807E0:0000009AE8180D58 0000009032df1aa0 0 MTA (Threadpool Completion Port) System.OutOfMemoryException 0000009ae8148908 (nested exceptions)
246 1265 3670 0000009f5eada010 80a9228 Preemptive 000000A8BFA16770:000000A8BFA16FD0 0000009032df1aa0 0 MTA (Threadpool Completion Port) System.OutOfMemoryException 000000a8bfa114b8 (nested exceptions)
272 1029 2c3c 0000009004ef0be0 8029228 Preemptive 0000009AE8CB9950:0000009AE8CBA460 0000009032df1aa0 0 MTA (Threadpool Completion Port) System.OutOfMemoryException 0000009ae8cb97f0
213 949 c4c 0000009fef212f00 a0228 Preemptive 0000009058538D40:000000905853AC20 0000009032df1aa0 0 MTA System.OutOfMemoryException 0000009058538c58
我已经过滤掉了(Threadpool Worker)和(Threadpool Completion Port)线程
Edit2更多信息: 我们一直在运行perfmon,我们可以看到的一件事是,在崩溃之前,正在使用的&#34; Sink块的数量&#34;发疯了它从稳定的200-300,000到600多万......有时它可以从中恢复,但有时它不能然后崩溃。
任何可能导致此问题的线索?
Edit3:添加了水槽锁的图像 https://imgur.com/a/bPNbh
Edit4:我们在服务器模式下使用GC运行
System.Runtime.GCSettings.LatencyMode = System.Runtime.GCLatencyMode.SustainedLowLatency;
的Henning