c3p0死锁检测 - 线程运行太频繁

时间:2013-06-26 18:27:37

标签: hibernate deadlock connection-pooling c3p0

我们正在使用Hibernate和c3p0连接池库。到目前为止,这个组合很有效,直到最近我们决定将maxPoolSize增加到1000并对我们的应用程序进行大量压力测试。我们的应用程序的峰值负载导致DB响应非常缓慢,因此c3p0的死锁检测器一次又一次地吐出APPARENT DEADLOCK警告。

基于c3p0文档,我们将maxAdministrativeTaskTime更改为10分钟,假设死锁检测每30分钟发生一次(代码表示死锁检测频率是maxAdministrativeTaskTime的三倍)。

但是,在分析c3p0日志时,死锁检测线程的运行频率超过30分钟。附加了日志的相关部分。令人惊讶的是,频率并不均匀。

Line 573745: [Timer-2] 2013-06-26 04:47:52,492 WARN [null] com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@662cee3b -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
Line 573746: [Timer-2] 2013-06-26 04:47:52,512 WARN [null] com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@662cee3b -- APPARENT DEADLOCK!!! Complete Status: 
Line 574292: [Timer-2] 2013-06-26 04:49:12,493 WARN [null] com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@662cee3b -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
Line 574293: [Timer-2] 2013-06-26 04:49:12,513 WARN [null] com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@662cee3b -- APPARENT DEADLOCK!!! Complete Status: 
Line 575004: [Timer-2] 2013-06-26 04:50:32,494 WARN [null] com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@662cee3b -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
Line 575005: [Timer-2] 2013-06-26 04:50:32,511 WARN [null] com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@662cee3b -- APPARENT DEADLOCK!!! Complete Status: 
Line 576062: [Timer-2] 2013-06-26 04:51:52,495 WARN [null] com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@662cee3b -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
Line 576063: [Timer-2] 2013-06-26 04:51:52,536 WARN [null] com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@662cee3b -- APPARENT DEADLOCK!!! Complete Status: 
Line 576720: [Timer-2] 2013-06-26 04:53:12,496 WARN [null] com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@662cee3b -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
Line 576721: [Timer-2] 2013-06-26 04:53:12,516 WARN [null] com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@662cee3b -- APPARENT DEADLOCK!!! Complete Status: 
Line 594087: [Timer-2] 2013-06-26 04:55:52,550 WARN [null] com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@662cee3b -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
Line 594088: [Timer-2] 2013-06-26 04:55:52,569 WARN [null] com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@662cee3b -- APPARENT DEADLOCK!!! Complete Status: 
Line 594753: [Timer-2] 2013-06-26 04:57:12,550 WARN [null] com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@662cee3b -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
Line 594754: [Timer-2] 2013-06-26 04:57:12,572 WARN [null] com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@662cee3b -- APPARENT DEADLOCK!!! Complete Status: 
Line 595624: [Timer-2] 2013-06-26 04:58:32,552 WARN [null] com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@662cee3b -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
Line 595625: [Timer-2] 2013-06-26 04:58:32,570 WARN [null] com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@662cee3b -- APPARENT DEADLOCK!!! Complete Status: 
Line 596416: [Timer-2] 2013-06-26 04:59:52,552 WARN [null] com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@662cee3b -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
Line 596417: [Timer-2] 2013-06-26 04:59:52,572 WARN [null] com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@662cee3b -- APPARENT DEADLOCK!!! Complete Status: 
Line 611011: [Timer-2] 2013-06-26 05:02:22,556 WARN [null] com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@662cee3b -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
Line 611012: [Timer-2] 2013-06-26 05:02:22,577 WARN [null] com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@662cee3b -- APPARENT DEADLOCK!!! Complete Status: 

有人可以解释这种异常吗?

2 个答案:

答案 0 :(得分:2)

maxAdministrativeTaskTime不会影响线程池的死锁检测器;它只是限制了在池尝试interrupt()之前允许单个任务执行的长度。一个( hackish!,丑陋!,不建议!)试图避免APPARENT DEADLOCKs的方法是将它设置为SHORT间隔,这样慢的任务就会被中断而不是挂起那么长的游泳池决定它被卡住了。请参阅一些讨论here

感谢提问者(通过电子邮件),我仔细检查了我自己的%$ ^ *&!代码和死锁检测间隔 maxAdministrativeTaskTime影响。如问题所示,它应该是3×maxAdministrativeTaskTime。所以神秘感加深了。

值得了解死锁检测器的作用。 c3p0维护一个线程池,numHelperThreads大。死锁检测器会定期记录池中的所有任务线程正在运行,然后休眠一段时间,然后再次检查。如果任务的 none 已完成,即活动任务与上次检查时保持完全相同,则会声明死锁。 c3p0期望某个任务完成的时间间隔为10秒,现在无法配置。它可以配置。

您可以发布一些您看到的APPARENT DEADLOCK的示例。 c3p0打印了大量信息,以帮助您了解可能会挂起的内容。

这里有两个建议在非常重的负载下减少APPARENT DEADLOCK:

  1. 您观察到与Connection acqusistion相关的死锁?如果是,如果您还没有,请升级到c3p0-0.9.2.1或最新的0.9.5预发行版。

  2. 您是否已经尝试增加numHelperThreads,因为您已经放大了负载?如果某些任务非常慢,除非所有线程被慢速任务阻止,否则c3p0不会声明死锁。如果有足够的线程使得更灵活的任务仍在执行,你就不会看到死锁。

  3. 我希望这有帮助!

答案 1 :(得分:1)

我无法解释,但1000个连接相当多,你的c3p0配置是否反映了这么多的连接?我的经验是,默认设置适用于较少数量的连接。 我建议阅读c3po documentation,尤其是“其他数据源配置”下的部分。