放置

Question

我们希望在springboot微服务中使用EHCache3来缓存二进制对象（PDF和一些元数据）这些对象介于50K到12m之间。使用简单的测试我可以加载缓存并看到它正常工作。

缓存大型PDF时会出现问题。显示以下堆栈跟踪

2017-02-03 21:08:47.936  INFO 11 --- [http-nio-8080-exec-4] .e.c.i.r.LoggingRobustResilienceStrategy : Ehcache key 5dd12ae803ea06c4d37afe949090b934 recovered from

org.ehcache.core.spi.store.StoreAccessException: java.lang.reflect.UndeclaredThrowableException
    at org.ehcache.core.exceptions.StorePassThroughException.handleRuntimeException(StorePassThroughException.java:72)
    at org.ehcache.impl.internal.store.offheap.AbstractOffHeapStore.computeWithRetry(AbstractOffHeapStore.java:1098)
    at org.ehcache.impl.internal.store.offheap.AbstractOffHeapStore.put(AbstractOffHeapStore.java:306)
    at org.ehcache.impl.internal.store.tiering.TieredStore.put(TieredStore.java:149)
    at org.ehcache.core.Ehcache.put(Ehcache.java:196)
    at com.pany.PdfGeneratorController.convertHtmlToPdf(PdfGeneratorController.java:79)
    at com.pany.microservice.pdfgenerator.endpoints.PdfGeneratorEndpoint.convertHtmlToPdfUsingPOST(PdfGeneratorEndpoint.java:23)
    at com.pany.microservice.pdfgenerator.api.PdfgeneratorcontrollerSpringEndpoint.convertHtmlToPdfUsingPOSTINNER(PdfgeneratorcontrollerSpringEndpoint.java:100)
    at com.pany.microservice.pdfgenerator.api.PdfgeneratorcontrollerSpringEndpoint.convertHtmlToPdfUsingPOSTPOST(PdfgeneratorcontrollerSpringEndpoint.java:67)
    at sun.reflect.GeneratedMethodAccessor137.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.__invoke(DelegatingMethodAccessorImpl.java:43)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:220)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:134)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:116)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:648)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
    at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.doFilterInternal(ApplicationContextHeaderFilter.java:55)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
    at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:105)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
    at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:89)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
    at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
    at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:107)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
    at org.springframework.boot.web.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:117)
    at org.springframework.boot.web.support.ErrorPageFilter.access$000(ErrorPageFilter.java:61)
    at org.springframework.boot.web.support.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:92)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.springframework.boot.web.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:110)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
    at org.apache.catalina.core.StandardContextValve.__invoke(StandardContextValve.java:108)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
    at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:783)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:789)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.UndeclaredThrowableException: null
    at com.sun.proxy.$Proxy122.encode(Unknown Source)
    at org.terracotta.offheapstore.storage.PortabilityBasedStorageEngine.writeMapping(PortabilityBasedStorageEngine.java:64)
    at org.terracotta.offheapstore.OffHeapHashMap.tryWriteEntry(OffHeapHashMap.java:703)
    at org.terracotta.offheapstore.OffHeapHashMap.writeEntry(OffHeapHashMap.java:687)
    at org.terracotta.offheapstore.OffHeapHashMap.computeWithMetadata(OffHeapHashMap.java:1947)
    at org.terracotta.offheapstore.AbstractLockedOffHeapHashMap.computeWithMetadata(AbstractLockedOffHeapHashMap.java:582)
    at org.terracotta.offheapstore.concurrent.AbstractConcurrentOffHeapMap.computeWithMetadata(AbstractConcurrentOffHeapMap.java:733)
    at org.ehcache.impl.internal.store.disk.EhcachePersistentConcurrentOffHeapClockCache.compute(EhcachePersistentConcurrentOffHeapClockCache.java:158)
    at org.ehcache.impl.internal.store.offheap.AbstractOffHeapStore.computeWithRetry(AbstractOffHeapStore.java:1083)
    ... 83 common frames omitted
Caused by: java.lang.reflect.InvocationTargetException: null
    at sun.reflect.GeneratedMethodAccessor136.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.__invoke(DelegatingMethodAccessorImpl.java:43)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.ehcache.impl.internal.store.disk.OffHeapDiskStore$1.invoke(OffHeapDiskStore.java:454)
    ... 92 common frames omitted
Caused by: java.lang.OutOfMemoryError: Java heap space

我怀疑问题在于我用于缓存的配置，但我无法弄明白。我在堆和磁盘存储中使用约束值来测试缓存已满时的正确驱逐。通常我希望新内容从缓存中推出旧内容，并降低性能。但是现在我失去了记忆异常，我无法解释。关于ehcache的分层还有其他的东西要理解吗？

<config
    xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
    xmlns='http://www.ehcache.org/v3'
    xmlns:jsr107='http://www.ehcache.org/v3/jsr107'
    xsi:schemaLocation="
        http://www.ehcache.org/v3 http://www.ehcache.org/schema/ehcache-core-3.1.xsd
        http://www.ehcache.org/v3/jsr107 http://www.ehcache.org/schema/ehcache-107-ext-3.1.xsd">

    <persistence directory="/root/cachexml"/>
    <heap-store>
        <max-object-size unit="MB">20</max-object-size>
    </heap-store>

    <cache alias="pdfCache" >
        <key-type>java.lang.String</key-type>
        <value-type>com.acquisio.microservice.pdfgenerator.msto.PDFTO</value-type>
        <expiry>
           <tti unit="hours">12</tti>
        </expiry>

         <resources>
             <heap unit="entries">4</heap>
             <disk persistent="true" unit="MB">64</disk>
        </resources>
    </cache>

</config>

Answer 1

在theheap上接受4个条目意味着可以有4个条目。无论大小。因此，如果您的堆很小并且您的对象很大，那么您确实可以获得OOME。

您的解决方案是为堆存储提供最大内存大小（class ImagesInline(admin.TabularInline): model = ProductGallery class ProductAdmin(admin.ModelAdmin): inlines = [ ImagesInline, ] admin.site.register(Product, ProductAdmin) ）而不是多个条目。

这样，条目将在需要时和获取OOME之前被驱逐（假设最大内存大小低于堆上剩余的内存）。

Answer 2

查看堆栈跟踪和Ehcache 3代码内部 - 大概是因为你没有给出确切的版本 - 似乎OOME在序列化值时发生，因此它可以存储在磁盘上。

当您在Ehcache 3中的多层缓存中放置一个值时，该值始终写入最低层 - 在您的情况下为磁盘。但是，为了做到这一点，必须序列化键和值。默认情况下，Ehcache 3依靠Java序列化来完成这项工作，因此需要键和值来实现Serializable，在PDFTO的情况下，如果您的设置有效，则需要{。}}。

详细说明，它的工作方式是对象被序列化然后放入ByteBuffer，然后通过offheap内存写入磁盘。但这意味着我们需要能够分配足够大的ByteBuffer才能继续。

所以问题不在于堆层，因为那里的对象在被传递到缓存之前已经被分配了，但是转移到了磁盘层。

我建议查看您的整个堆使用情况，并了解您为何如此受限制，或者检查对象的序列化大小。如果他们有太多传出的引用，可能会有一个惊喜潜伏在那里。