我正在使用Camel 2.18和Java8。下面是一些带有路由的代码。我将此类JSON发布到此服务:
{
"bigField": "testtesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttest",
"smallField": "234"
}
问题是bigField很大,可以为256MB。那就是为什么我想到在Camel中进行流缓存:http://camel.apache.org/stream-caching.html不知道是否可以在场景中应用它?我了解的是,这种机制应该一点一点地将流存储在磁盘上吗?我创建了/ tmp / cachedir,然后看到Camel如何创建和删除一些临时文件。
6:15:23.505 [qtp422330142-21 - /myservice/v1/oper] DEBUG org.apache.camel.util.FileUtil - Retrying attempt 0 to delete file: /tmp/cachedir/cos7138988885969104934.tmp
16:15:23.505 [qtp422330142-21 - /myservice/v1/oper] DEBUG org.apache.camel.util.FileUtil - Tried 1 to delete file: /tmp/cachedir/cos7138988885969104934.tmp with result: true
16:15:23.505 [qtp422330142-21 - /myservice/v1/oper] DEBUG org.apache.camel.util.FileUtil - Retrying attempt 0 to delete file: /tmp/cachedir/cos8802932624903146810.tmp
16:15:23.505 [qtp422330142-21 - /myservice/v1/oper] DEBUG org.apache.camel.util.FileUtil - Tried 1 to delete file: /tmp/cachedir/cos8802932624903146810.tmp with result: true
16:15:23.505 [qtp422330142-21 - /myservice/v1/oper] DEBUG org.apache.camel.util.FileUtil - Retrying attempt 0 to delete file: /tmp/cachedir/cos91165405800652292.tmp
16:15:23.505 [qtp422330142-21 - /myservice/v1/oper] DEBUG org.apache.camel.util.FileUtil - Tried 1 to delete file: /tmp/cachedir/cos91165405800652292.tmp with result: true
16:15:23.506 [qtp422330142-21 - /myservice/v1/oper] DEBUG org.apache.camel.util.FileUtil - Retrying attempt 0 to delete file: /tmp/cachedir/cos6540772182497948066.tmp
16:15:23.506 [qtp422330142-21 - /myservice/v1/oper] DEBUG org.apache.camel.util.FileUtil - Tried 1 to delete file: /tmp/cachedir/cos6540772182497948066.tmp with result: true
16:15:23.506 [qtp422330142-21 - /myservice/v1/oper] DEBUG org.eclipse.jetty.server.Server - RESPONSE /myservice/v1/oper 201
所以它似乎正在工作。但是我正在使用这种机制来保护自己免受OutOfMemory异常的影响。所以我写了Junit发送如下创建的bigField:
headers.setAll(map);
char[] chars = new char[20000000];
Arrays.fill(chars, 'a');
String bigField = new String(chars);
我使用JAVA_OPTS="-Xms60m -Xmx128m"
我缺少什么?我以为这种流缓存机制可以保护我免受客户端发送大于Xmx-128MB的正文事件的侵害,但失败了20MB。
@SpringBootApplication
public class Application extends RouteBuilder {
@Override
public void configure() throws Exception {
setupStreamCaching();
restConfiguration().host("0.0.0.0").port(PORT)
.endpointProperty("headerFilterStrategy", "#myHeaderFilterStrategy")
.endpointProperty("matchOnUriPrefix", "true")
.endpointProperty("sendServerVersion", "false")
.bindingMode(RestBindingMode.json);
rest(API_CONTEXT + V1)
.post(API_OPERATION)
.type(RequestModel.class)
.outType(Response.class)
.consumes(APPLICATION_JSON)
.produces(APPLICATION_JSON)
.to(MAIN_ROUTE);
from(MAIN_ROUTE)
.routeId(MAIN_ROUTE)
.process(requestValidator)
.to(SERVICE_CALL)
.end();
private void setupStreamCaching() {
getContext().getStreamCachingStrategy().setSpoolDirectory("/tmp/cachedir");
getContext().getStreamCachingStrategy().setSpoolThreshold(64 * 1024);
getContext().getStreamCachingStrategy().setBufferSize(16 * 1024);
}
http://camel.apache.org/stream-caching.html
3:30:24.584 [qtp1492219097-24-/ myservice / v1 / oper]调试 org.eclipse.jetty.io.ChannelEndPoint-填充3432 SelectChannelEndPoint @ 44b2fa12 {/10.0.2.2:57125 <-> 1080,打开,输入,输出,-,-,0/30000,HttpConnection} {io = 0,kio = 0,kro = 1} 13:30:24.585 [qtp1492219097-24-/ myservice / v1 / oper]调试 o.e.jetty.server.HttpConnection- HttpConnection @ 13dbbbcc [REFILLING,SelectChannelEndPoint @ 44b2fa12 {/10.0.2.2:57125 <-> 1080,Open,in,out,-,-,0/30000,HttpConnection} {io = 0,kio = 0,kro = 1} ] [p = HttpParser {s = CONTENT,19996672 的 20000104},g = HttpGenerator {s = START},c = HttpChannelOverHttp @ 4b6953ae {r = 1,c = false,a = DISPATCHED,uri = / myservice / v1 / oper}] 填充3432 13:30:24.585 [qtp1492219097-24-/ myservice / v1 / oper]调试 org.eclipse.jetty.http.HttpParser-parseNext s =内容 HeapByteBuffer @ 42f6e192 [p = 0,l = 3432,c = 8192,r = 3432] = {<< >> aaaaaaaaaaaaaaaaa ... aaaaaaaaaaaaaaaaa} 13:30:24.585 [qtp1492219097-24-/ myservice / v1 / oper]调试 org.eclipse.jetty.server.HttpChannel- HttpChannelOverHttp @ 4b6953ae {r = 1,c = false,a = DISPATCHED,uri = / myservice / v1 / oper} 内容java.nio.HeapByteBufferR [pos = 0 lim = 3432 cap = 8192] 13:30:24.585 [qtp1492219097-24-/ myservice / v1 / oper]调试 org.eclipse.jetty.http.HttpParser-parseNext s =内容 HeapByteBuffer @ 42f6e192 [p = 3432,l = 3432,c = 8192,r = 0] = {aaaaaaaaaaaaaaaaaaaaaaa ...- 21T17:32:28Z“} <<< >>> aaaaaaaaaaaaaaaaa ... aaaaaaaaaaaaaaaaaaa} 13:30:24.585 [qtp1492219097-24-/ myservice / v1 / oper]调试 org.eclipse.jetty.http.HttpParser-目录->结束13:30:24.585 [qtp1492219097-24-/ myservice / v1 / oper]调试 org.eclipse.jetty.server.HttpChannel- HttpChannelOverHttp @ 4b6953ae {r = 1,c = false,a = DISPATCHED,uri = / myservice / v1 / oper} messageComplete 13:30:24.585 [qtp1492219097-24-/ myservice / v1 / oper] 调试org.eclipse.jetty.server.HttpInput-HttpInputOverHTTP @ 63602aee EOF 13:30:24.585 [qtp1492219097-24-/ myservice / v1 / oper]调试 org.eclipse.jetty.io.ChannelEndPoint-填充0 SelectChannelEndPoint @ 44b2fa12 {/10.0.2.2:57125 <-> 1080,Open,in,out,-,-,1/30000,HttpConnection} {io = 0,kio = 0,kro = 1} 13:30:24.586 [qtp1492219097-24-/ myservice / v1 / oper]调试 o.e.jetty.server.HttpConnection- HttpConnection @ 13dbbbcc [REFILLING,SelectChannelEndPoint @ 44b2fa12 {/10.0.2.2:57125 <-> 1080,Open,in,out,-,-,2/30000,HttpConnection} {io = 0,kio = 0,kro = 1} ] [p = HttpParser {s = END,20000104 的 20000104},g = HttpGenerator {s = START},c = HttpChannelOverHttp @ 4b6953ae {r = 1,c = false,a = DISPATCHED,uri = / myservice / v1 / oper}] 填充0 13:30:24.586 [qtp1492219097-24-/ myservice / v1 / oper]调试 org.eclipse.jetty.server.HttpInput-HttpInputOverHTTP @ 63602aee eof EOF 13:30:24.586 [qtp1492219097-24-/ myservice / v1 / oper]调试 org.eclipse.jetty.server.HttpInput-HttpInputOverHTTP @ 63602aee eof EOF 13:30:27.462 [qtp1492219097-24-/ myservice / v1 / oper]警告 o.e.jetty.servlet.ServletHandler-/ myservice / v1 / oper org.apache.camel.TypeConversionException:类型转换期间出错 从类型:java.lang.String到所需的类型:java.lang.String 值[Body是org.apache.camel.StreamCache的实例] java.lang.OutOfMemoryError:Java堆空间
java.lang.OutOfMemoryError:Java堆空间位于 org.apache.camel.impl.converter.BaseTypeConverterRegistry.createTypeConversionException(BaseTypeConverterRegistry.java:629) 在 org.apache.camel.impl.converter.BaseTypeConverterRegistry.convertTo(BaseTypeConverterRegistry.java:150) 在 org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:78) 在 org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:53) 在 org.apache.camel.util.MessageHelper.extractBodyAsString(MessageHelper.java:84) 在 org.apache.camel.component.rest.RestConsumerBindingProcessor.process(RestConsumerBindingProcessor.java:162) 在 org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77) 在 org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:542) 在 org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197) 在org.apache.camel.processor.Pipeline.process(Pipeline.java:120)处 org.apache.camel.processor.Pipeline.process(Pipeline.java:83)在 org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197) 在 org.apache.camel.component.jetty.CamelContinuationServlet.doService(CamelContinuationServlet.java:191) 在 org.apache.camel.http.common.CamelServlet.service(CamelServlet.java:74) 在javax.servlet.http.HttpServlet.service(HttpServlet.java:790)处 org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) 在 org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587) 在 org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) 在 org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) 在 org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) 在 org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) 在 org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) 在org.eclipse.jetty.server.Server.handle(Server.java:499)处 org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)在 org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) 在 org.eclipse.jetty.io.AbstractConnection $ 2.run(AbstractConnection.java:544) 在 org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) 在 org.eclipse.jetty.util.thread.QueuedThreadPool $ 3.run(QueuedThreadPool.java:555) 在java.lang.Thread.run(Thread.java:748)造成原因: org.apache.camel.RuntimeCamelException:java.lang.OutOfMemoryError: Java堆空间位于 org.apache.camel.util.ObjectHelper.wrapRuntimeCamelException(ObjectHelper.java:1763) 在 org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:1358) 在 org.apache.camel.impl.converter.StaticMethodTypeConverter.convertTo(StaticMethodTypeConverter.java:59) 在 org.apache.camel.impl.converter.BaseTypeConverterRegistry.doConvertTo(BaseTypeConverterRegistry.java:306) 在 org.apache.camel.impl.converter.BaseTypeConverterRegistry.convertTo(BaseTypeConverterRegistry.java:133) ...省略了27个共同的框架
由以下原因引起:java.lang.OutOfMemoryError:Java堆空间位于 java.util.Arrays.copyOf(Arrays.java:3332)在 java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124) 在 java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:596) 在java.lang.StringBuilder.append(StringBuilder.java:190)在 org.apache.camel.converter.IOConverter.toString(IOConverter.java:318) 在 org.apache.camel.converter.IOConverter.toString(IOConverter.java:307) 在 org.apache.camel.converter.IOConverter.toString(IOConverter.java:364) 在sun.reflect.NativeMethodAccessorImpl.invoke0(本机方法)处 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 在 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 在java.lang.reflect.Method.invoke(Method.java:498)在 org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:1354) 在 org.apache.camel.impl.converter.StaticMethodTypeConverter.convertTo(StaticMethodTypeConverter.java:59) 在 org.apache.camel.impl.converter.BaseTypeConverterRegistry.doConvertTo(BaseTypeConverterRegistry.java:306) 在 org.apache.camel.impl.converter.BaseTypeConverterRegistry.convertTo(BaseTypeConverterRegistry.java:133) 在 org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:78) 在 org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:53) 在 org.apache.camel.util.MessageHelper.extractBodyAsString(MessageHelper.java:84) 在 org.apache.camel.component.rest.RestConsumerBindingProcessor.process(RestConsumerBindingProcessor.java:162) 在 org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77) 在 org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:542) 在 org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197) 在org.apache.camel.processor.Pipeline.process(Pipeline.java:120)处 org.apache.camel.processor.Pipeline.process(Pipeline.java:83)在 org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197) 在 org.apache.camel.component.jetty.CamelContinuationServlet.doService(CamelContinuationServlet.java:191) 在 org.apache.camel.http.common.CamelServlet.service(CamelServlet.java:74) 在javax.servlet.http.HttpServlet.service(HttpServlet.java:790)处 org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) 在 org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587) 在 org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)