Ehcache到期和死锁

时间:2015-09-09 16:26:41

标签: performance hibernate spring-data-jpa ehcache

我使用的是hibernate 4.2.20和ehcache 2.6.6。

我有2个实体和一个弹簧数据存储库,如下所示:

@Entity
@Inheritance(strategy = InheritanceType.TABLE_PER_CLASS)
@Cache(region = "branch", usage = CacheConcurrencyStrategy.TRANSACTIONAL)
public abstract class AbstractBranch{
 .....
}

@Entity
@Table(name = "branch")
public class Branch extends AbstractBranch {
}

public interface BranchRepository extends BaseRepository<Branch, UUID> {
     @QueryHints({ @QueryHint(name = "org.hibernate.cacheable", value = "true"), @QueryHint(name = "org.hibernate.cacheRegion", value = "branch") })
     Branch findByBranchId(String branchId);
}

在我的ehcache.xml中,我对分支缓存有以下内容:

<cache name="branch" maxEntriesLocalHeap="10000" eternal="false" timeToLiveSeconds="1800" statistics="true" transactionalMode="xa">

现在我正在尝试运行性能测试,一切都运行良好,直到此缓存过期:

一旦过期,多个线程调用取景器:

branchRepository.findByBranchId(...).

我开始获得TimeOut / Deadlock异常。见stracktrace be

TimeoutManage I   WTRN0124I: When the timeout occurred the thread with which the transaction is, or was most recently, associated was Thread[WebContainer : 96,5,main]. The stack trace of this thread when the timeout occurred was: 
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:197)
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:845)
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:975)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1293)
java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:742)
net.sf.ehcache.store.FrontEndCacheTier.containsKeyInMemory(FrontEndCacheTier.java:483)
net.sf.ehcache.transaction.AbstractTransactionStore.containsKeyInMemory(AbstractTransactionStore.java:187)
net.sf.ehcache.transaction.AbstractTransactionStore.containsKeyInMemory(AbstractTransactionStore.java:187)
net.sf.ehcache.Cache.searchInStoreWithStats(Cache.java:1941)
net.sf.ehcache.Cache.get(Cache.java:1584)
org.hibernate.cache.ehcache.internal.regions.EhcacheGeneralDataRegion.get(EhcacheGeneralDataRegion.java:74)
org.hibernate.cache.ehcache.internal.regions.EhcacheQueryResultsRegion.get(EhcacheQueryResultsRegion.java:39)
org.hibernate.cache.internal.StandardQueryCache.getCachedResults(StandardQueryCache.java:201)
org.hibernate.cache.internal.StandardQueryCache.get(StandardQueryCache.java:140)
org.hibernate.loader.Loader.getResultFromQueryCache(Loader.java:2477)
org.hibernate.loader.Loader.listUsingQueryCache(Loader.java:2385)
org.hibernate.loader.Loader.list(Loader.java:2358)
org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:495)
org.hibernate.hql.internal.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:357)
org.hibernate.engine.query.spi.HQLQueryPlan.performList(HQLQueryPlan.java:198)
org.hibernate.internal.SessionImpl.list(SessionImpl.java:1230)
org.hibernate.internal.QueryImpl.list(QueryImpl.java:101)
org.hibernate.ejb.QueryImpl.getSingleResult(QueryImpl.java:287)
org.hibernate.ejb.criteria.CriteriaQueryCompiler$3.getSingleResult(CriteriaQueryCompiler.java:258)
org.springframework.data.jpa.repository.query.JpaQueryExecution$SingleEntityExecution.doExecute(JpaQueryExecution.java:206)
org.springframework.data.jpa.repository.query.JpaQueryExecution.execute(JpaQueryExecution.java:78)
org.springframework.data.jpa.repository.query.AbstractJpaQuery.doExecute(AbstractJpaQuery.java:100)
org.springframework.data.jpa.repository.query.AbstractJpaQuery.execute(AbstractJpaQuery.java:91)
org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.doInvoke(RepositoryFactorySupport.java:413)
org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.invoke(RepositoryFactorySupport.java:391)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:281)
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:136)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodIntercceptor.invoke(CrudMethodMetadataPostProcessor.java:122)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
com.sun.proxy.$Proxy125.findByBranchId(Unknown Source)

我在https://gist.github.com/seamusmac/84843bce06bd0e4fd2f0发布了一个帖子转储 它显示了5个被阻止的线程。

有人请问,如何防止这种情况发生?

2 个答案:

答案 0 :(得分:1)

根据EhCache文档,由于您在ehcache.xml中指定的XA事务模式,这些似乎可能是“正常”的超时异常......

此类XA模式将在更新,删除或添加期间使用写锁定。并且通过并发访问,这些锁会导致某些线程阻塞并显示死锁。最终死锁线程超时(并抛出异常)以避免陷入死锁状态。

请查看此页面http://ehcache.org/documentation/2.6/apis/transactions#transaction-managers,在“常见问题解答”部分的“为什么有些线程会定期超时并抛出异常?”的问题中明确提到这一点。

答案 1 :(得分:0)

可以通过在LocalTransactionStore:put中设置断点来重现此问题。如果缓存已过期,则触发第一个请求,执行将在put中停止。然后触发第二个请求,该请求将被删除并等待原始锁定。重新启动第一个线程,你就会遇到死锁。

以下是我认为它锁定的原因:

线程1 :(堆栈已锁定)     private void doAcquireShared(int arg) -> int r = tryAcquireShared(arg); -> else if (current == getExclusiveOwnerThread())

java.util.concurrent.locks.AbstractOwnableSynchronizer.setExclusiveOwnerThread(AbstractOwnableSynchronizer.java:56) java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryAcquire(ReentrantReadWriteLock.java:411) java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1245) java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.tryLock(ReentrantReadWriteLock.java:1115) net.sf.ehcache.concurrent.ReadWriteLockSync.tryLock(ReadWriteLockSync.java:57) net.sf.ehcache.Cache.tryRemoveImmediately(Cache.java:2103) net.sf.ehcache.Cache.searchInStoreWithStats(Cache.java:1963) net.sf.ehcache.Cache.get(Cache.java:1584) org.hibernate.cache.ehcache.internal.strategy.TransactionalEhcacheEntityRegionAccessStrategy.putFromLoad(TransactionalEhcacheEntityRegionAccessStrategy.java:122) org.hibernate.cache.ehcache.internal.nonstop.NonstopAwareEntityRegionAccessStrategy.putFromLoad(NonstopAwareEntityRegionAccessStrategy.java:195)

主题2: (堆栈设置独占所有者线程)     Unsafe.park(boolean, long) line: not available [native method] LockSupport.parkNanos(Object, long) line: 226 ReentrantLock$NonfairSync(AbstractQueuedSynchronizer).doAcquireNanos(int, long) line: 929 ReentrantLock$NonfairSync(AbstractQueuedSynchronizer).tryAcquireNanos(int, long) line: 1245 ReentrantLock.tryLock(long, TimeUnit) line: 445 ReadCommittedSoftLockImpl.tryLock(long) line: 85 LocalTransactionStore.remove(Object) line: 467 JtaLocalTransactionStore.remove(Object) line: 371 Cache.removeInternal(Object, boolean, boolean, boolean, boolean) line: 2334 Cache.tryRemoveImmediately(Object, boolean) line: 2117
Cache.searchInStoreWithStats(Object) line: 1963 Cache.get(Object) line: 1584
TransactionalEhcacheEntityRegionAccessStrategy.putFromLoad(Object, Object, long, Object, boolean) line: 122

(堆叠锁定) protected final boolean tryAcquire(int acquires) { /* * Walkthrough: * 1. If read count nonzero or write count nonzero * and owner is a different thread, fail. * 2. If count would saturate, fail. (This can only * happen if count is already nonzero.) * 3. Otherwise, this thread is eligible for lock if * it is either a reentrant acquire or * queue policy allows it. If so, update state * and set owner. */ Thread current = Thread.currentThread(); int c = getState(); int w = exclusiveCount(c); if (c != 0) { // (Note: if c != 0 and w == 0 then shared count != 0) if (w == 0 || current != getExclusiveOwnerThread()) return false; if (w + exclusiveCount(acquires) > MAX_COUNT) throw new Error("Maximum lock count exceeded"); // Reentrant acquire setState(c + acquires); return true; } if (writerShouldBlock() || !compareAndSetState(c, c + acquires)) return false; setExclusiveOwnerThread(current); return true; }

第二个帖子setExclusiveOwnerThread作为其中一部分:tryAcquireNanos - &gt; tryAcquire(arg)然后将公园作为其中一部分:tryAcquireNanos - &gt; doAcquireNanos(arg,nanosTimeout)因此,它不会注册读取或写入锁定,但它会使用独占所有者设置保留锁定。

从这一点开始,第一个线程中对ReadLock.lock的任何调用都将停止,因为tryAcquireShared失败,因为current == getExclusiveOwnerThread()为false,导致死锁。

现在问题是:为什么ReadLock.lock()会停止独占所有者线程集?

更新:扩展堆栈我已经意识到事件发生在net.sf.ehcache.Cache的不同部分并使用不同的类:

Cache.tryRemoveImmediately(Cache.java:2103) - &gt; ReadWriteLockSync.tryLock(ReadWriteLockSync.java:57)

Cache.tryRemoveImmediately(Object,boolean)line:2117 - &gt; (...) - &gt; ReentrantLock.tryLock(long,TimeUnit)

第一个设置独占所有者线程并在tryAcquire中返回true。

的ReentrantReadWriteLock $ Sync.tryAcquire ReentrantLock$Sync.tryAcquire return nonfairTryAcquire(acquires); final boolean nonfairTryAcquire(int acquires) { final Thread current = Thread.currentThread(); int c = getState(); if (c == 0) { if (compareAndSetState(0, acquires)) { setExclusiveOwnerThread(current); return true; } } else if (current == getExclusiveOwnerThread()) { int nextc = c + acquires; if (nextc < 0) // overflow throw new Error("Maximum lock count exceeded"); setState(nextc); return true; } return false; }

第二个错误,导致doAcquireNanos执行并停车     onOptionsItemSelected(MenuItem item)