我们发现了应用程序的阻塞问题,并且从堆栈转储中发现,许多线程(其中一个是catalina-exec-998)通过锁0x000000065888b918被线程catalina-exec-995阻塞,并且我们还发现阻塞程序线程catalina-exec-995也处于BLOCKED状态,似乎它试图获取该对象监视器已被另一个线程拥有,但是在堆栈调用中没有有关该对象监视器地址的信息它试图等待,
所以我的问题是如何找到等待其锁定的对象地址?
另外,如何通过JVM选项-XX:CompileCommand防止JIT编译对象监视器以使等待信息始终可用?
顺便说一句,我在java-thread-dump-waiting-on-object-monitor-what-is-it-waiting-on中也发现了类似的问题,从JVM专家的回答中,我知道丢失的“等待中”信息可能是由JIT编译器引起的,该URL中的解决方案仅适合同步对象中的Object.wait(),但是我的处境明显不同,因为它试图获取对象监视器而不是等待从同步对象中获取通知。
以下信息是提到的两个线程的调用堆栈:
"catalina-exec-995" #1178 daemon prio=5 os_prio=0 tid=0x00007f6b7831f000 nid=0x15c0 waiting for monitor entry [0x00007f6b43ffc000]
java.lang.Thread.State: BLOCKED (on object monitor)
at java.util.zip.ZipFile.getEntry(ZipFile.java:308)
- locked <**0x000000065888b918**> (a java.util.jar.JarFile)
at java.util.jar.JarFile.getEntry(JarFile.java:240)
at java.util.jar.JarFile.getJarEntry(JarFile.java:223)
at org.apache.catalina.webresources.AbstractSingleArchiveResourceSet.getArchiveEntry(AbstractSingleArchiveResourceSet.java:94)
at org.apache.catalina.webresources.AbstractArchiveResourceSet.getResource(AbstractArchiveResourceSet.java:260)
at org.apache.catalina.webresources.StandardRoot.getResourcesInternal(StandardRoot.java:327)
at org.apache.catalina.webresources.CachedResource.validateResources(CachedResource.java:124)
- locked <0x000000068aa3c7a0> (a org.apache.catalina.webresources.CachedResource)
at org.apache.catalina.webresources.Cache.getResources(Cache.java:147)
at org.apache.catalina.webresources.StandardRoot.getResources(StandardRoot.java:315)
at org.apache.catalina.webresources.StandardRoot.getClassLoaderResources(StandardRoot.java:231)
at org.apache.catalina.loader.WebappClassLoaderBase.findResources(WebappClassLoaderBase.java:939)
at java.lang.ClassLoader.getResources(ClassLoader.java:1142)
at com.sun.jersey.spi.service.ServiceFinder.getResources(ServiceFinder.java:352)
at com.sun.jersey.spi.service.ServiceFinder.access$200(ServiceFinder.java:159)
at com.sun.jersey.spi.service.ServiceFinder$AbstractLazyIterator.setConfigs(ServiceFinder.java:720)
at com.sun.jersey.spi.service.ServiceFinder$AbstractLazyIterator.hasNext(ServiceFinder.java:732)
at com.sun.jersey.spi.service.ServiceFinder.toClassArray(ServiceFinder.java:595)
at com.sun.jersey.core.spi.component.ProviderServices.getServiceClasses(ProviderServices.java:318)
at com.sun.jersey.core.spi.component.ProviderServices.getServiceClasses(ProviderServices.java:311)
at com.sun.jersey.core.spi.component.ProviderServices.getServices(ProviderServices.java:159)
at com.sun.jersey.core.spi.factory.MessageBodyFactory.initWriters(MessageBodyFactory.java:207)
at com.sun.jersey.core.spi.factory.MessageBodyFactory.init(MessageBodyFactory.java:163)
at com.sun.jersey.api.client.Client.init(Client.java:342)
at com.sun.jersey.api.client.Client.access$000(Client.java:118)
at com.sun.jersey.api.client.Client$1.f(Client.java:191)
at com.sun.jersey.api.client.Client$1.f(Client.java:187)
at com.sun.jersey.spi.inject.Errors.processWithErrors(Errors.java:193)
at com.sun.jersey.api.client.Client.<init>(Client.java:187)
at com.sun.jersey.api.client.Client.<init>(Client.java:170)
at com.sun.jersey.api.client.Client.create(Client.java:679)
at com.meritdata.tempo.foundation.security.util.RestfulUtils.request(RestfulUtils.java:33)
at com.meritdata.tempo.foundation.security.util.RestfulUtils.requestGet(RestfulUtils.java:28)
at com.meritdata.tempo.foundation.security.util.PaWebUtils.validatePaToken(PaWebUtils.java:344)
at com.meritdata.tempo.foundation.security.sso.PaSSO.readUserToken(PaSSO.java:137)
at com.meritdata.tempo.foundation.security.filter.UserLoginFilter.validatePortalLogin(UserLoginFilter.java:190)
at com.meritdata.tempo.foundation.security.filter.UserLoginFilter.onAccessDenied(UserLoginFilter.java:248)
at org.apache.shiro.web.filter.AccessControlFilter.onAccessDenied(AccessControlFilter.java:133)
at org.apache.shiro.web.filter.AccessControlFilter.onPreHandle(AccessControlFilter.java:162)
at org.apache.shiro.web.filter.PathMatchingFilter.isFilterChainContinued(PathMatchingFilter.java:203)
at org.apache.shiro.web.filter.PathMatchingFilter.preHandle(PathMatchingFilter.java:178)
at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:131)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383)
at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:344)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:261)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.meritdata.tempo.foundation.security.filter.XssFilter.doFilter(XssFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.meritdata.tempo.foundation.security.filter.CacheControlFilter.doFilter(CacheControlFilter.java:41)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.orm.hibernate4.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:152)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
- locked <0x0000000612c3a940> (a org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Locked ownable synchronizers:
- <0x000000062d3e9728> (a java.util.concurrent.ThreadPoolExecutor$Worker)
"catalina-exec-998" #1181 daemon prio=5 os_prio=0 tid=0x00007f6b7c359800 nid=0x15c3 waiting for monitor entry [0x00007f6b43cf9000]
java.lang.Thread.State: BLOCKED (on object monitor)
at java.util.zip.ZipFile.getEntry(ZipFile.java:308)
- waiting to lock <**0x000000065888b918**> (a java.util.jar.JarFile)
at java.util.jar.JarFile.getEntry(JarFile.java:240)
at java.util.jar.JarFile.getJarEntry(JarFile.java:223)
at org.apache.catalina.webresources.AbstractSingleArchiveResourceSet.getArchiveEntry(AbstractSingleArchiveResourceSet.java:94)
at org.apache.catalina.webresources.AbstractArchiveResourceSet.getResource(AbstractArchiveResourceSet.java:260)
at org.apache.catalina.webresources.StandardRoot.getResourcesInternal(StandardRoot.java:327)
at org.apache.catalina.webresources.CachedResource.validateResources(CachedResource.java:124)
- locked <0x0000000691ae3b90> (a org.apache.catalina.webresources.CachedResource)
at org.apache.catalina.webresources.Cache.getResources(Cache.java:147)
at org.apache.catalina.webresources.StandardRoot.getResources(StandardRoot.java:315)
at org.apache.catalina.webresources.StandardRoot.getClassLoaderResources(StandardRoot.java:231)
at org.apache.catalina.loader.WebappClassLoaderBase.findResources(WebappClassLoaderBase.java:939)
at java.lang.ClassLoader.getResources(ClassLoader.java:1142)
at com.sun.jersey.spi.service.ServiceFinder.getResources(ServiceFinder.java:352)
at com.sun.jersey.spi.service.ServiceFinder.access$200(ServiceFinder.java:159)
at com.sun.jersey.spi.service.ServiceFinder$AbstractLazyIterator.setConfigs(ServiceFinder.java:720)
at com.sun.jersey.spi.service.ServiceFinder$AbstractLazyIterator.hasNext(ServiceFinder.java:732)
at com.sun.jersey.spi.service.ServiceFinder.toClassArray(ServiceFinder.java:595)
at com.sun.jersey.core.spi.component.ProviderServices.getServiceClasses(ProviderServices.java:318)
at com.sun.jersey.core.spi.component.ProviderServices.getServiceClasses(ProviderServices.java:311)
at com.sun.jersey.core.spi.component.ProviderServices.getServices(ProviderServices.java:159)
at com.sun.jersey.core.spi.factory.MessageBodyFactory.initWriters(MessageBodyFactory.java:207)
at com.sun.jersey.core.spi.factory.MessageBodyFactory.init(MessageBodyFactory.java:163)
at com.sun.jersey.api.client.Client.init(Client.java:342)
at com.sun.jersey.api.client.Client.access$000(Client.java:118)
at com.sun.jersey.api.client.Client$1.f(Client.java:191)
at com.sun.jersey.api.client.Client$1.f(Client.java:187)
at com.sun.jersey.spi.inject.Errors.processWithErrors(Errors.java:193)
at com.sun.jersey.api.client.Client.<init>(Client.java:187)
at com.sun.jersey.api.client.Client.<init>(Client.java:170)
at com.sun.jersey.api.client.Client.create(Client.java:679)
at com.meritdata.tempo.foundation.security.util.RestfulUtils.request(RestfulUtils.java:33)
at com.meritdata.tempo.foundation.security.util.RestfulUtils.requestGet(RestfulUtils.java:28)
at com.meritdata.tempo.foundation.security.util.PaWebUtils.validatePaToken(PaWebUtils.java:344)
at com.meritdata.tempo.foundation.security.filter.UserLoginFilter.keepPaTokenInCookie(UserLoginFilter.java:496)
at com.meritdata.tempo.foundation.security.filter.UserLoginFilter.isAccessAllowed(UserLoginFilter.java:448)
at org.apache.shiro.web.filter.AccessControlFilter.onPreHandle(AccessControlFilter.java:162)
at org.apache.shiro.web.filter.PathMatchingFilter.isFilterChainContinued(PathMatchingFilter.java:203)
at org.apache.shiro.web.filter.PathMatchingFilter.preHandle(PathMatchingFilter.java:178)
at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:131)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383)
at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:344)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:261)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.meritdata.tempo.foundation.security.filter.XssFilter.doFilter(XssFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.meritdata.tempo.foundation.security.filter.CacheControlFilter.doFilter(CacheControlFilter.java:41)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.orm.hibernate4.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:152)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
- locked <0x0000000612c9dee0> (a org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Locked ownable synchronizers:
- <0x000000062ca8ffc8> (a java.util.concurrent.ThreadPoolExecutor$Worker)
答案 0 :(得分:0)
在您的情况下,catalina-exec-995
和catalina-exec-998
线程竞争同一个对象监视器-地址为JarFile
的{{1}}实例。 stacktrace已经具有所有信息。
您可能对以下事实感到困惑:两个线程都处于0x000000065888b918
状态,而其中一个线程是BLOCKED
监视器,而另一个则拥有waiting to lock
这个监视器。这是绝对正常的情况:获取对象监视器和切换线程状态并不是原子发生的。
正在等待首先进入对象监视器的线程 获取锁,然后从locked
切换到BLOCKED
状态。当您在两者之间进行线程转储时,您可能会完全了解所看到的情况。
让我通过一个简单的示例进行演示。
RUNNABLE
线程转储可能看起来像
public class Sync implements Runnable {
int n;
public static void main(String[] args) {
Sync sync = new Sync();
new Thread(sync, "Sync1").start();
new Thread(sync, "Sync2").start();
new Thread(sync, "Sync3").start();
}
public void run() {
for (;;) {
inc();
}
}
public synchronized void inc() {
n++;
}
}
很明显,只有一个对象监视器-"Sync3" #13 prio=5 os_prio=0 tid=0x000000001deab800 nid=0x25d4 runnable [0x000000001e98f000]
java.lang.Thread.State: RUNNABLE
at Sync.run(Sync.java:13)
at java.lang.Thread.run(Thread.java:748)
"Sync2" #12 prio=5 os_prio=0 tid=0x000000001deab000 nid=0x230c waiting for monitor entry [0x000000001e88f000]
java.lang.Thread.State: BLOCKED (on object monitor)
at Sync.inc(Sync.java:18)
- locked <0x000000076b8a1108> (a Sync)
at Sync.run(Sync.java:13)
at java.lang.Thread.run(Thread.java:748)
"Sync1" #11 prio=5 os_prio=0 tid=0x000000001deaa000 nid=0x16d4 waiting for monitor entry [0x000000001e78f000]
java.lang.Thread.State: BLOCKED (on object monitor)
at Sync.inc(Sync.java:18)
- waiting to lock <0x000000076b8a1108> (a Sync)
at Sync.run(Sync.java:13)
at java.lang.Thread.run(Thread.java:748)
的单个实例。 Sync
和Sync1
线程都是Sync2
,但是其中一个线程是BLOCKED
,另一个线程有waiting to lock
个监视器。