我们有一个由 SQL Server 2008 R2 数据库（Hibernate 3 ORM - C3P0 0.9.2.1 ）支持的java（1.6.x）Web应用程序，通常几次当系统负载很重时，我们会在日志中看到这些行，并且客户抱怨速度很慢。
2016-03-01 10:11:27,372 WARN C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-AdminTaskTimer com.mchange.v2.async.ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@4c0bee48 -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
2016-03-01 10:11:27,372 WARN C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-AdminTaskTimer com.mchange.v2.async.ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@4c0bee48 -- APPARENT DEADLOCK!!! Complete Status:
Managed Threads: 20
Active Threads: 20
Active Tasks:
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask@69de8a09
on thread: C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#11
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask@11388b2c
on thread: C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#3
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask@1738fe89
on thread: C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#1
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask@1ecee048
on thread: C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#13
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask@1fe6a6c5
on thread: C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#2
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask@5fcedfdf
on thread: C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#10
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask@6cbda9d
on thread: C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#0
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask@317d7ce7
on thread: C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#4
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask@25933b13
on thread: C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#12
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask@650b6faa
on thread: C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#5
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask@51d035b9
on thread: C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#17
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask@3c772ea5
on thread: C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#9
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask@349771a6
on thread: C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#14
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask@42a66443
on thread: C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#15
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask@3a008ce4
on thread: C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#7
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask@7a21ea7b
on thread: C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#16
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask@48a96fe1
on thread: C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#6
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask@6bf0420b
on thread: C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#18
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask@6a801789
on thread: C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#19
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask@7f8959fe
on thread: C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#8
Pending Tasks:
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@1273741c
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@59cbe3ba
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@58d3f664
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@43d54ccb
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@68a4c637
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@19d17a8d
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@7c4cc39
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@93bb62b
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@2f34c5c9
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@31a52755
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@51b956e9
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@5ef904e9
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@2eb18db5
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@ace7e06
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@7ab3c174
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@2f61e59b
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@44293b6a
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@6efbcf1b
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@55da2e89
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@67255067
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@42c1cf18
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@420352e5
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@68375a3e
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@a05d1e6
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@417582e
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@189e361b
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@3af2333d
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@73020806
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@a5cfce7
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@5edfa701
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@2d4d6350
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@33b3306c
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@39301cf8
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@5bdfb1d
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@7bb12fdc
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@2dbd53f5
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@68d31148
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@7913c3bb
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@976817
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@5432be4e
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@4f6c1e11
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@432a16d3
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@409918
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@1102f6c9
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@59980ed8
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@df6b4dc
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@3fdbdead
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@7bf8509b
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@6b742fb2
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@18e7863b
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@81d9843
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@4ef75ade
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@516ecd52
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@4169ec42
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@1116029b
com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask@3c0516e0
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@7220eae8
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@4f1e5fe1
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@5328d55b
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@1dcf7404
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@1ec9c1e6
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@4fd8f2d4
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@2c2e6540
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@19f96369
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@43ed8fd1
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@217d052e
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@16df202f
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@12b9f964
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@6fa41dd6
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@7fa305dc
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@27d5effb
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@4e7e5c2b
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@49d5334f
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@4c1bad67
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@2d156c3a
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@59746af5
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@6a1a28b4
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@5b6677d7
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@2445f322
Pool thread stack traces:
Thread[C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#11,5,main]
com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1380)
com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:179)
com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.closePreparedHandle(SQLServerPreparedStatement.java:148)
com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.closeInternal(SQLServerPreparedStatement.java:175)
com.microsoft.sqlserver.jdbc.SQLServerStatement.close(SQLServerStatement.java:595)
com.mchange.v1.db.sql.StatementUtils.attemptClose(StatementUtils.java:53)
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask.run(GooGooStatementCache.java:938)
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:648)
Thread[C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#3,5,main]
com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1380)
com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:179)
com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.closePreparedHandle(SQLServerPreparedStatement.java:148)
com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.closeInternal(SQLServerPreparedStatement.java:175)
com.microsoft.sqlserver.jdbc.SQLServerStatement.close(SQLServerStatement.java:595)
com.mchange.v1.db.sql.StatementUtils.attemptClose(StatementUtils.java:53)
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask.run(GooGooStatementCache.java:938)
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:648)
Thread[C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#13,5,main]
com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1380)
com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:179)
com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.closePreparedHandle(SQLServerPreparedStatement.java:148)
com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.closeInternal(SQLServerPreparedStatement.java:175)
com.microsoft.sqlserver.jdbc.SQLServerStatement.close(SQLServerStatement.java:595)
com.mchange.v1.db.sql.StatementUtils.attemptClose(StatementUtils.java:53)
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask.run(GooGooStatementCache.java:938)
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:648)
Thread[C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#1,5,main]
com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1380)
com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:179)
com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.closePreparedHandle(SQLServerPreparedStatement.java:148)
com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.closeInternal(SQLServerPreparedStatement.java:175)
com.microsoft.sqlserver.jdbc.SQLServerStatement.close(SQLServerStatement.java:595)
com.mchange.v1.db.sql.StatementUtils.attemptClose(StatementUtils.java:53)
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask.run(GooGooStatementCache.java:938)
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:648)
Thread[C3P0PooledConnectionPoolManager[identityToken->1bqoobh9fc5bldvr10ww5|13e9d0cc]-HelperThread-#2,5,main]
com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1380)
com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:179)
com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.closePreparedHandle(SQLServerPreparedStatement.java:148)
com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.closeInternal(SQLServerPreparedStatement.java:175)
com.microsoft.sqlserver.jdbc.SQLServerStatement.close(SQLServerStatement.java:595)
com.mchange.v1.db.sql.StatementUtils.attemptClose(StatementUtils.java:53)
com.mchange.v2.c3p0.stmt.GooGooStatementCache$StatementDestructionManager$1UncheckedStatementCloseTask.run(GooGooStatementCache.java:938)
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:648)
我们的池连接设置如下：
<property name="maxIdleTime" value="600" />
<property name="maxPoolSize" value="200" />
<property name="maxStatements" value="200" />
<property name="minPoolSize" value="15" />
<property name="idleConnectionTestPeriod" value="100" />
<property name="acquireIncrement" value="3"/>
<property name="numHelperThreads" value="20"/>
我们应该调查什么？