Question

我正在调查在Java 7上运行的Glassfish 3.1.2.2托管应用程序遇到的CPU使用率高的问题

CPU使用率将在20分钟的过程中从“正常”水平开始上升，从5-10％上升到100％，然后保持在90-100之间并且不会下降，重新启动应用程序将恢复正常状态。

以下2个摘录摘自2个线程转储，它们之间的间隔为10分钟。

使用visualvm CPU profiler快照交叉引用整个转储，显示大约有10个线程执行此代码区域，这些线程似乎在下面的方法中花费了全部时间。

除了锁定对象引用之外，所有10个线程的两个线程转储中的整个堆栈（以下是巨大的缩写）都是相同的。

我想知道这里发生了什么。包括为什么锁定对象引用已更改？

代码是卡在循环中还是在某个地方有锁？

线程转储1

"http-thread-pool-8080(3)" - Thread t@112
   java.lang.Thread.State: RUNNABLE
    at java.lang.Throwable.fillInStackTrace(Native Method)
    at java.lang.Throwable.fillInStackTrace(Throwable.java:783)
    - locked <2328e584> (a java.lang.InterruptedException)
    at java.lang.Throwable.<init>(Throwable.java:250)
    at java.lang.Exception.<init>(Exception.java:54)
    at java.lang.InterruptedException.<init>(InterruptedException.java:57)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1325)
    at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.tryLock(ReentrantReadWriteLock.java:873)
    at com.sun.corba.ee.impl.oa.poa.POAImpl.acquireLock(POAImpl.java:390)
    at com.sun.corba.ee.impl.oa.poa.POAImpl.readLock(POAImpl.java:422)
    at com.sun.corba.ee.impl.oa.poa.POAImpl.enter(POAImpl.java:1743)
    at com.sun.corba.ee.impl.protocol.CorbaServerRequestDispatcherImpl.getServantWithPI(CorbaServerRequestDispatcherImpl.java:302)
    at com.sun.corba.ee.impl.protocol.CorbaServerRequestDispatcherImpl.dispatch(CorbaServerRequestDispatcherImpl.java:196)
    at com.sun.corba.ee.impl.protocol.CorbaMessageMediatorImpl.handleRequestRequest(CorbaMessageMediatorImpl.java:1624)
    at com.sun.corba.ee.impl.protocol.SharedCDRClientRequestDispatcherImpl.marshalingComplete(SharedCDRClientRequestDispatcherImpl.java:126)
    at com.sun.corba.ee.impl.protocol.CorbaClientDelegateImpl.invoke(CorbaClientDelegateImpl.java:273)
    at com.sun.corba.ee.impl.protocol.CorbaClientDelegateImpl.is_a(CorbaClientDelegateImpl.java:395)
    at org.omg.CORBA.portable.ObjectImpl._is_a(ObjectImpl.java:130)
...

线程转储2

"http-thread-pool-8080(3)" - Thread t@112
   java.lang.Thread.State: RUNNABLE
    at java.lang.Throwable.fillInStackTrace(Native Method)
    at java.lang.Throwable.fillInStackTrace(Throwable.java:783)
    - locked <83c9c3a> (a java.lang.InterruptedException)
    at java.lang.Throwable.<init>(Throwable.java:250)
    at java.lang.Exception.<init>(Exception.java:54)
    at java.lang.InterruptedException.<init>(InterruptedException.java:57)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1325)
    at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.tryLock(ReentrantReadWriteLock.java:873)
    at com.sun.corba.ee.impl.oa.poa.POAImpl.acquireLock(POAImpl.java:390)
    at com.sun.corba.ee.impl.oa.poa.POAImpl.readLock(POAImpl.java:422)
    at com.sun.corba.ee.impl.oa.poa.POAImpl.enter(POAImpl.java:1743)
    at com.sun.corba.ee.impl.protocol.CorbaServerRequestDispatcherImpl.getServantWithPI(CorbaServerRequestDispatcherImpl.java:302)
    at com.sun.corba.ee.impl.protocol.CorbaServerRequestDispatcherImpl.dispatch(CorbaServerRequestDispatcherImpl.java:196)
    at com.sun.corba.ee.impl.protocol.CorbaMessageMediatorImpl.handleRequestRequest(CorbaMessageMediatorImpl.java:1624)
    at com.sun.corba.ee.impl.protocol.SharedCDRClientRequestDispatcherImpl.marshalingComplete(SharedCDRClientRequestDispatcherImpl.java:126)
    at com.sun.corba.ee.impl.protocol.CorbaClientDelegateImpl.invoke(CorbaClientDelegateImpl.java:273)
    at com.sun.corba.ee.impl.protocol.CorbaClientDelegateImpl.is_a(CorbaClientDelegateImpl.java:395)
    at org.omg.CORBA.portable.ObjectImpl._is_a(ObjectImpl.java:130)
...

更新

这可能是导致问题的POAImpl acquisitionLock方法...

private void acquireLock(Lock lock) {
        MethodMonitor __$mm$__ = (MethodMonitor)__$mm$__0.content();
        if (__$mm$__ != null) {
            __$mm$__.enter(1, new Object[]{lock});
        }

        try {
            long timeout = 1L;
            boolean locked = false;
            boolean interrupted = false;
            int count = 0;
            int reportingThreshhold = 1;

            while(!locked) {
                if (count >= reportingThreshhold) {
                    this.acquireLockWaiting(count, __$mm$__, 1);
                    if (reportingThreshhold < 1073741823) {
                        reportingThreshhold *= 2;
                    }
                }

                try {
                    locked = lock.tryLock(1L, TimeUnit.SECONDS);
                    ++count;
                } catch (InterruptedException var13) {
                    interrupted = true;
                }

                if (interrupted) {
                    Thread.currentThread().interrupt();
                }
            }

            if (__$mm$__ != null) {
                __$mm$__.exit(1);
            }

        } finally {
            if (__$mm$__ != null) {
                __$mm$__.exit(1);
            }

        }
    }

Answer 1

一种选择是在代码中发生类似的事情：

public void doTheJob(){

     try{
       .... // fail fast for whatever reason
     }catch(Exception e){
        doTheJob();
     }
}

发生异常-这就是为什么我们看到stacktracke的原因。您会陷入无限循环。

非常有趣的是，堆栈跟踪中有一个InterruptedException，因此看起来您可能试图杀死一些未完成的（定时）任务，并可能重新安排它们的时间。

Answer 2

关于提供的代码。其本质上是这样的：

public static void main(String[] args) throws InterruptedException {
    ReentrantLock lock = new ReentrantLock();
    Thread t=new Thread(()->lock.lock()); //lets simulate that Lock is locked
    t.start();
    t.join();
    int times = 0;
    Thread.currentThread().interrupt(); //and for whatever reasons - thread was interrupted from outside
    boolean locked=false;
    while (!locked) {
        try {
            boolean gotLock=lock.tryLock(1, TimeUnit.SECONDS);
            System.out.println("Got lock?: "+gotLock);
        } catch (InterruptedException e) {
            System.out.println("Thrown times:" + times++);
            Thread.currentThread().interrupt(); // iterrupts again - will throw on getLock no matter what now
        }
    }
}

因此基本上一次中断-您将陷入无限循环的困境-没有延迟-这会占用CPU。我建议围绕该中断处理添加诊断日志记录，以查看发生了什么情况。

可运行线程-是否已移动？

2 个答案: