Question

我们在集群Weblogic环境中运行带有Hibernate的SpringMVC。我们的应用程序运行良好，直到我们遇到数据库空间问题。因此，我们的数据库提供程序扩展了数据库存储。

在更改之后，Weblogic系统状态正在抱怨线程池“线程池已经卡住线程”。

日志文件中的堆栈跟踪如下所示：

<Error> <WebLogicServer> <BEA-000337> <[STUCK] ExecuteThread: '12' for queue: 'weblogic.ke
rnel.Default (self-tuning)' has been busy for "641" seconds working on the request "weblogic.servlet.internal.ServletReq
uestImpl@17782ff3[
GET /default/default/applicationPreloader/load.do?_=1331887527968 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:10.0.2) Gecko/20100101 Firefox/10.0.2
Accept: */*
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip, deflate
x-requested-with: XMLHttpRequest
Referer: http://novintegweb01.eu.acme.net:9136/default/default/applicationPreloader.do
Cookie: JSESSIONID1=NhBdPv9Hx8Wj5lnr5vnGhLSbR2GGM1DPhqT2YGzWCGFvhN3WJWVH!700169023!1391220923
Connection: Keep-Alive
Proxy-Client-IP: 192.168.1.1
X-Forwarded-For: 192.168.1.1
X-WebLogic-KeepAliveSecs: 30
X-WebLogic-Request-ClusterInfo: true
x-weblogic-cluster-hash: G4kGT1kfuDBbtj5w//SYNt6hlVo

]", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
        java.net.SocketInputStream.socketRead0(Native Method)
        java.net.SocketInputStream.read(SocketInputStream.java:129)
        oracle.net.ns.Packet.receive(Packet.java:293)
        oracle.net.ns.DataPacket.receive(DataPacket.java:104)
        oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:315)
        oracle.net.ns.NetInputStream.read(NetInputStream.java:260)
        oracle.net.ns.NetInputStream.read(NetInputStream.java:185)
        oracle.net.ns.NetInputStream.read(NetInputStream.java:102)
        oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:124)
        oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:80)
        oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1136)
        oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1113)
        oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:288)
        oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:191)
        oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:523)
        oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:207)
        oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:1009)
        oracle.jdbc.driver.OraclePreparedStatement.executeForRowsWithTimeout(OraclePreparedStatement.java:10117)
        oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:10221)
        oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:216)
        weblogic.jdbc.wrapper.PreparedStatement.executeBatch(PreparedStatement.java:210)
        org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:48)
        org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:247)
        org.hibernate.jdbc.AbstractBatcher.prepareStatement(AbstractBatcher.java:93)
        org.hibernate.jdbc.AbstractBatcher.prepareStatement(AbstractBatcher.java:88)
        org.hibernate.jdbc.AbstractBatcher.prepareBatchStatement(AbstractBatcher.java:223)
        org.hibernate.persister.collection.AbstractCollectionPersister.recreate(AbstractCollectionPersister.java:1119)
        org.hibernate.action.CollectionRecreateAction.execute(CollectionRecreateAction.java:39)
        org.hibernate.engine.ActionQueue.execute(ActionQueue.java:279)
        org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:263)
        org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:171)
        org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:298)
        org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:27)
        org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1001)
        org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:339)
        org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:106)
        org.springframework.orm.hibernate3.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:656)
        org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:754)
        org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723)
        org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:393)
        org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:120)
        org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
        org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)
        org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
        org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:621)
        com.acme.superproject.dao.CertificationDaoImpl$$EnhancerByCGLIB$$bf3135a5.getById(<generated>)
        com.acme.superproject.integration.UserProfileSabaServiceImpl.determineTargetDateFromWs(UserProfileSabaServiceImpl.java:436)
        com.acme.superproject.integration.UserProfileSabaServiceImpl.determineTargetDate(UserProfileSabaServiceImpl.java:401)
        com.acme.superproject.integration.UserProfileSabaServiceImpl.createCertificationRelationObject(UserProfileSabaServiceImpl.java:373)
        com.acme.superproject.integration.UserProfileSabaServiceImpl.getRequiredLearning(UserProfileSabaServiceImpl.java:334)
        com.acme.superproject.integration.UserProfileSabaServiceImpl.extractUserProfile(UserProfileSabaServiceImpl.java:203)
        com.acme.superproject.integration.UserProfileSabaServiceImpl.getUserProfile(UserProfileSabaServiceImpl.java:73)
        com.acme.superproject.integration.UserProfileSabaServiceImpl.getUserProfile(UserProfileSabaServiceImpl.java:114)
        com.acme.superproject.integration.UserProfileSabaServiceImpl$$FastClassByCGLIB$$e48c61c9.invoke(<generated>)
        net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149)
        org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:688)
        org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
        org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80)
        com.acme.superproject.techarch.TracingAspect.trace(TracingAspect.java:31)
        sun.reflect.GeneratedMethodAccessor5382.invoke(Unknown Source)
        sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        java.lang.reflect.Method.invoke(Method.java:597)
        org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
        org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
        org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
        org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
        org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)
        org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
        org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:621)
        com.acme.superproject.integration.UserProfileSabaServiceImpl$$EnhancerByCGLIB$$5d7b85df.getUserProfile(<generated>)
        com.acme.superproject.service.UserProfileServiceImpl.getUserProfile(UserProfileServiceImpl.java:26)
        com.acme.superproject.service.UserProfileServiceImpl$$FastClassByCGLIB$$70cfe815.invoke(<generated>)
        net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149)
        org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:688)
        org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
        org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80)
        com.acme.superproject.techarch.TracingAspect.trace(TracingAspect.java:31)
        sun.reflect.GeneratedMethodAccessor5382.invoke(Unknown Source)
        sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        java.lang.reflect.Method.invoke(Method.java:597)
        org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
        org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
        org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
        org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
        org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)
        org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
        org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:621)
        com.acme.superproject.service.UserProfileServiceImpl$$EnhancerByCGLIB$$5bcb212b.getUserProfile(<generated>)
        com.acme.superproject.web.session.SessionDataProvider.fetchUserProfile(SessionDataProvider.java:97)
        com.acme.superproject.web.session.SessionDataProvider.getUserProfile(SessionDataProvider.java:78)
        com.acme.superproject.web.session.SessionDataProvider.fetchUserRegistrationCollection(SessionDataProvider.java:155)
        com.acme.superproject.web.session.SessionDataProvider.getUserRegistrationCollection(SessionDataProvider.java:141)
        com.acme.superproject.web.ApplicationPreloaderController.loadAction(ApplicationPreloaderController.java:84)
        sun.reflect.GeneratedMethodAccessor6506.invoke(Unknown Source)
        sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        java.lang.reflect.Method.invoke(Method.java:597)
        org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:176)
        org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:426)
        org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:414)
        org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:790)
        org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
        org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
        org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:549)
        javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
        javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
        weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
        weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
        weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:300)
        weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
        weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:56)
        org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:368)
        org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:109)
        org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83)
        org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:380)
        org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:97)
        org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:380)
        org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105)
        org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:380)
        org.springframework.security.web.authentication.preauth.AbstractPreAuthenticatedProcessingFilter.doFilter(AbstractPreAuthenticatedProcessingFilter.java:89)
        org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:380)
        org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:79)
        org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:380)
        org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:169)
        org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
        org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
        weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:56)
        org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
        org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
        weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:56)
        weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3715)
        weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3681)
        weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
        weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
        weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2277)
        weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2183)
        weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1454)
        weblogic.work.ExecuteThread.execute(ExecuteThread.java:207)
        weblogic.work.ExecuteThread.run(ExecuteThread.java:176)
>

我们已经重新启动了Weblogic，数据库服务器并进行了新的部署。问题总是一样的：当我们尝试访问应用程序的登录页面时，它会挂起 - 就我在尝试执行数据库事务时所看到的那样。

除此之外，当我尝试使用“Oracle SQL Developer”连接到我们的数据库时，我们也会收到错误“ORA-00257：archiver error。仅连接内部，直到释放”。不幸的是，我们没有SYSDBA权限（因为由外部提供商管理）。

我的问题是：

你们中有谁有过类似的问题吗？
您是否认为此类问题的根本原因在于应用程序方面，或者更多在DB方面？
我怎样才能至少摆脱被困的线程？

Answer 1

数据库存储已满，或者归档程序无法移动或创建新的存档日志。这几乎肯定是数据库级别的问题，而不是应用程序服务器级别。由于您收到此消息表示您已经连接到数据库，因此它不会坐着等待连接。

WebLogic中的Stuck Threads并不总是需要“修复”，并不总是一个问题。卡住线程机制的目的是在线程花费异常长时间做某事时（例如访问数据库）通知您。卡住的线程通知只是 - 通知某些内容可能不正确，并且在这种情况下它正在服务于此目的。

如果您的线程通常需要20分钟才能执行某些操作但希望在超过25分钟时收到通知，则必须将Stuck Thread Limit增加到25分钟并在此时获得通知然后您必须将此限制从600秒提高到1500秒（25 * 60）。

您可以在weblogic admin base_domain＆gt;中编辑卡住的线程限制。环境＆gt;服务器＆gt;管理服务器＆gt;配置/调整。

参考以下链接： https://docs.oracle.com/cd/E19316-01/820-3745/ghudn/index.html

Answer 2

这似乎是数据库的一个问题。线程卡在DB上。你的表现似乎已经降低了。我建议将此案件提交给您的DBA进行进一步调查。另外，尝试在正在访问的表上重新创建索引。

Answer 3

当Oracle DB Server在开发人员环境中的VirtualBox上时，我们遇到了这个问题。指出这个问题有点困难。当ThreadPool没有任何线程来处理请求并且请求被拒绝时，我们注意到了这一点。

Weblogic和Oracle：Stuck ExecuteThread

3 个答案: