我有一个简单的休息服务,它从Mysql db获取数据。我正在使用tomcat。当我运行250 TPS测试时,cpu使用率上升到~85%。我看看哪个java线程正在消耗更多的内存,并且基于我从线程转储中查看线程,但无法弄清楚发生了什么。
我在tomcat线程池中有2个核心cpu,200个线程。
Tasks: 202 total, 2 running, 200 sleeping, 0 stopped, 0 zombie
Cpu(s): 5.0%us, 2.2%sy, 0.0%ni, 92.5%id, 0.2%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 7539820k total, 7409136k used, 130684k free, 74432k buffers
Swap: 0k total, 0k used, 0k free, 5031940k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
24589 root 20 0 5551m 1.8g 16m S 3.9 25.0 0:06.33 java
24998 root 20 0 5551m 1.8g 16m S 3.9 25.0 0:00.75 java
25043 root 20 0 5551m 1.8g 16m S 3.9 25.0 0:00.74 java
25051 root 20 0 5551m 1.8g 16m S 3.9 25.0 0:00.73 java
18407 root 20 0 5551m 1.8g 16m S 2.0 25.0 1:02.71 java
19413 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:50.43 java
24385 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:17.01 java
24386 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:16.84 java
24441 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:12.61 java
24443 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:12.51 java
24540 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.33 java
24543 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.37 java
24552 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.26 java
24554 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.21 java
24555 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.37 java
24556 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.31 java
24558 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.20 java
24560 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.27 java
24562 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.28 java
24563 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.32 java
24564 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.30 java
24566 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.33 java
24568 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.20 java
24569 root 20 0 5551m 1.8g 16m R 2.0 25.0 0:06.21 java
24573 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.24 java
24575 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.24 java
24577 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.25 java
24580 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.36 java
24581 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.29 java
24583 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.32 java
24585 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.25 java
24591 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.32 java
24592 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.31 java
24595 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.30 java
24600 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.33 java
24604 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.33 java
24605 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.36 java
24610 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.41 java
24616 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.28 java
24618 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.37 java
24620 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.25 java
24623 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.20 java
24624 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:06.33 java
24999 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:00.71 java
25000 root 20 0 5551m 1.8g 16m R 2.0 25.0 0:00.70 java
25001 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:00.76 java
25006 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:00.74 java
25007 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:00.73 java
25013 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:00.74 java
25017 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:00.76 java
25029 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:00.69 java
25038 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:00.73 java
25039 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:00.74 java
25046 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:00.74 java
25047 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:00.69 java
25050 root 20 0 5551m 1.8g 16m S 2.0 25.0 0:00.73 java
12594 root 20 0 5551m 1.8g 16m S 0.0 25.0 0:00.00 java
12595 root 20 0 5551m 1.8g 16m S 0.0 25.0 0:19.58 java
12596 root 20 0 5551m 1.8g 16m S 0.0 25.0 0:08.76 java
12597 root 20 0 5551m 1.8g 16m S 0.0 25.0 0:08.83 java
12598 root 20 0 5551m 1.8g 16m S 0.0 25.0 0:14.77 java
12599 root 20 0 5551m 1.8g 16m S 0.0 25.0 0:00.18 java
12600 root 20 0 5551m 1.8g 16m S 0.0 25.0 0:00.16 java
12601 root 20 0 5551m 1.8g 16m S 0.0 25.0 0:00.00 java
12604 root 20 0 5551m 1.8g 16m S 0.0 25.0 0:00.00 java
12605 root 20 0 5551m 1.8g 16m S 0.0 25.0 0:00.01 java
12606 root 20 0 5551m 1.8g 16m S 0.0 25.0 0:00.00 java
12607 root 20 0 5551m 1.8g 16m S 0.0 25.0 0:01.02 java
线程转储:
"http-apr-8443-exec-363" #397 daemon prio=5 os_prio=0 tid=0x00007f97cc2d0000 nid=0x600f waiting on condition [0x00007f978cbd2000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000722096050> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:85)
at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:31)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
"http-apr-8443-exec-362" #396 daemon prio=5 os_prio=0 tid=0x00007f97cc1b2000 nid=0x600e waiting on condition [0x00007f978cdd4000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000722096050> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:85)
at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:31)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745Connection to 52.32.230.38 closed by remote host.
Connection to 52.32.230.38 closed.
qadeersmsiphone:/ pdubey$ 395 daemon prio=5 os_prio=0 tid=0x00007f97cc254800 nid=0x600d waiting on condition [0x00007f978d5dc000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000722096050> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:85)
at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:31)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
"http-apr-8443-exec-360" #394 daemon prio=5 os_prio=0 tid=0x00007f97cc063000 nid=0x600c waiting on condition [0x00007f978d6dd000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000722096050> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:85)
at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:31)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
"http-apr-8443-exec-359" #393 daemon prio=5 os_prio=0 tid=0x00007f97cc072000 nid=0x600b waiting on condition [0x00007f978d8df000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000722096050> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:85)
1794,41-48 55%
顶线程序(PID 24589):
http-apr-8443-exec-361" #395 daemon prio=5 os_prio=0 tid=0x00007f97cc254800 nid=0x600d waiting on condition [0x00007f978d5dc000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000722096050> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:85)
at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:31)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
什么可能导致如此高的CPU使用率。通常,当负载较小时,Web服务响应约为30 ms。
答案 0 :(得分:0)
竞争2个处理器内核的200个线程可能导致上下文切换占用大部分时间并提高处理器使用率。降低池大小并再次测试。