Question

我们正在Eclipse中测试我们的Spring Integration应用程序。

应用程序轮询数据库，使用rowMapper创建域对象，然后将这些对象编组为XML（使用Castor），并将XML作为消息下游发送。该应用程序运行良好，但在某些时候我们得到堆错误。

VM参数设置如下：-Xms512M -Xmx1024M

当轮询器必须处理非常大的结果集时，会出现抛出的OutOfMemoryExceptions。

这是轮询器的配置：

<task:executor id="inboundAdapterPollerPool" pool-size="1000 queue-capacity="100000" />
<int-jdbc:inbound-channel-adapter id="jdbcPollingChannelAdapter"
   channel="input" auto-startup="true"
   query="${sql}" row-mapper="entryRowMapper" update=""
   select-sql-parameter-source="timeRangeSqlParameterSource"
   jdbc-operations="myJdbcTemplate">
   <int:poller fixed-rate="50" task-executor="inboundAdapterPollerPool" error-channel="error" >
      <int:transactional transaction-manager="transactionManager" isolation="DEFAULT"  timeout="-1" synchronization-factory="syncFactory"/>
   </int:poller>
</int-jdbc:inbound-channel-adapter>

以下是在不同执行期间抛出的两个例外：

INFO  org.springframework.integration.endpoint.SourcePollingChannelAdapter: started jdbcPollingChannelAdapter
INFO  org.springframework.context.support.DefaultLifecycleProcessor: Starting beans in phase -2147483648
INFO  org.springframework.context.support.DefaultLifecycleProcessor: Starting beans in phase 2147483647
ERROR org.springframework.integration.handler.LoggingHandler: org.springframework.core.task.TaskRejectedException: Executor [java.util.concurrent.ThreadPoolExecutor@1ac364e] did not accept task: org.springframework.integration.util.ErrorHandlingTaskExecutor$1@1215a83
    at org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor.execute(ThreadPoolTaskExecutor.java:244)
    at org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:49)
    at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:231)
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:53)
    at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
Caused by: java.util.concurrent.RejectedExecutionException
    at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.reject(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.execute(Unknown Source)
    at org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor.execute(ThreadPoolTaskExecutor.java:241)
    ... 12 more

和

INFO  org.springframework.integration.endpoint.SourcePollingChannelAdapter: started jdbcPollingChannelAdapter
INFO  org.springframework.context.support.DefaultLifecycleProcessor: Starting beans in phase -2147483648
INFO  org.springframework.context.support.DefaultLifecycleProcessor: Starting beans in phase 2147483647
ERROR org.springframework.transaction.support.TransactionSynchronizationUtils: TransactionSynchronization.afterCompletion threw exception
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Unknown Source)
    at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
    at java.lang.AbstractStringBuilder.append(Unknown Source)
    at java.lang.StringBuilder.append(Unknown Source)
    at org.springframework.integration.message.GenericMessage.toString(GenericMessage.java:82)
    at java.lang.String.valueOf(Unknown Source)
    at java.lang.StringBuilder.append(Unknown Source)
    at org.springframework.integration.transaction.ExpressionEvaluatingTransactionSynchronizationProcessor.doProcess(ExpressionEvaluatingTransactionSynchronizationProcessor.java:107)
    at org.springframework.integration.transaction.ExpressionEvaluatingTransactionSynchronizationProcessor.processAfterRollback(ExpressionEvaluatingTransactionSynchronizationProcessor.java:99)
    at org.springframework.integration.transaction.DefaultTransactionSynchronizationFactory$DefaultTransactionalResourceSynchronization.afterCompletion(DefaultTransactionSynchronizationFactory.java:93)
    at org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCompletion(TransactionSynchronizationUtils.java:168)
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.invokeAfterCompletion(AbstractPlatformTransactionManager.java:993)
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCompletion(AbstractPlatformTransactionManager.java:968)
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.processRollback(AbstractPlatformTransactionManager.java:872)
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.rollback(AbstractPlatformTransactionManager.java:822)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.completeTransactionAfterThrowing(TransactionAspectSupport.java:410)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:114)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
    at $Proxy15.call(Unknown Source)
    at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:236)
    at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:52)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
ERROR org.springframework.transaction.support.TransactionSynchronizationUtils: TransactionSynchronization.afterCompletion threw exception
java.lang.OutOfMemoryError: Java heap space
ERROR org.springframework.transaction.support.TransactionSynchronizationUtils: TransactionSynchronization.afterCompletion threw exception
java.lang.OutOfMemoryError: Java heap space

适配器或轮询器配置是否有任何明显错误，或者我可以做些什么来克服内存问题

非常感谢

Answer 1

嗯，你的应用程序显然无法跟上每50毫秒轮询的速度。

您可以尝试将拒绝策略设置为CALLER_RUNS但这会导致消息无序处理（当队列已满时，轮询器线程将处理任何新消息）但至少会限制轮询器。

您还可以编写一个自定义RejectedExecutionHandler来阻止轮询器线程，直到队列中有空间为止。

那就是说，你的队列大小相当大。

最简单的修复方法可能是将轮询频率降低到下游流程可以跟上的值。

Answer 2

似乎是您的任务执行程序配置的问题，您已将线程池和队列配置为非常大的值“100000”。

建议使用一些较小的值来调整它以避免内存问题。

<task:executor id="inboundAdapterPollerPool" pool-size="100-500" queue-capacity="1000" rejection-policy="CALLER_RUNS" />

默认情况下，拒绝策略为AbortPolicy，如果队列已满，则会抛出异常。但是，如果您需要在高负载下限制任务，可以使用CallerRunsPolicy。这允许执行程序“赶上”它正在处理的任务，从而释放队列，池中或两者中的一些容量。

Jdbc适配器/轮询器在重负载下似乎耗尽内存

2 个答案: