如何处理RedisMessageListenerContainer死亡

时间:2019-04-11 16:48:26

标签: spring-data-redis

我遇到了一个情况,我的春季启动应用程序中的redis pubsub RedisMessageListenerContainer死于

ERROR .RedisMessageListenerContainer: SubscriptionTask aborted with exception:
 org.springframework.dao.QueryTimeoutException: Redis command timed out; nested exception is com.lambdaworks.redis.RedisCommandTimeoutException: Command timed out
  at org.springframework.data.redis.connection.lettuce.LettuceExceptionConverter.convert(LettuceExceptionConverter.java:66)
  at org.springframework.data.redis.connection.lettuce.LettuceExceptionConverter.convert(LettuceExceptionConverter.java:41)
  at org.springframework.data.redis.PassThroughExceptionTranslationStrategy.translate(PassThroughExceptionTranslationStrategy.java:37)
  at org.springframework.data.redis.FallbackExceptionTranslationStrategy.translate(FallbackExceptionTranslationStrategy.java:37)
  at org.springframework.data.redis.connection.lettuce.LettuceConnection.convertLettuceAccessException(LettuceConnection.java:330)
  at org.springframework.data.redis.connection.lettuce.LettuceConnection.subscribe(LettuceConnection.java:3179)
  at org.springframework.data.redis.listener.RedisMessageListenerContainer$SubscriptionTask.eventuallyPerformSubscription(RedisMessageListenerContainer.java:790)
  at org.springframework.data.redis.listener.RedisMessageListenerContainer$SubscriptionTask.run(RedisMessageListenerContainer.java:746)
  at java.lang.Thread.run(Thread.java:748)
 Caused by: com.lambdaworks.redis.RedisCommandTimeoutException: Command timed out
  at com.lambdaworks.redis.LettuceFutures.await(LettuceFutures.java:113)
  at com.lambdaworks.redis.LettuceFutures.awaitOrCancel(LettuceFutures.java:92)
  at com.lambdaworks.redis.FutureSyncInvocationHandler.handleInvocation(FutureSyncInvocationHandler.java:63)
  at com.lambdaworks.redis.internal.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:80)
  at com.sun.proxy.$Proxy156.subscribe(Unknown Source)
  at org.springframework.data.redis.connection.lettuce.LettuceSubscription.doSubscribe(LettuceSubscription.java:63)
  at org.springframework.data.redis.connection.util.AbstractSubscription.subscribe(AbstractSubscription.java:142)
  at org.springframework.data.redis.connection.lettuce.LettuceConnection.subscribe(LettuceConnection.java:3176)
  ... 3 common frames omitted
..

我认为这首先不是一个不可恢复的错误,因为它是一个临时连接问题(和TransientDataAccessException),但是在这种情况下,应用程序显然需要处理异常。

当前,这会使应用程序处于无法接受的状态。它只是记录错误,但是我要么需要杀死应用程序以便将其替换,要么更好地尝试重新启动该容器,并理想地通过/health报告该应用程序受到了影响,只要它不是很好。

是否有什么我想忽略的事情比尝试每隔x秒start()容器或对其子类化并覆盖handleSubscriptionException()并尝试从那里行动更尴尬?后者需要与我的代码进行更深入的内部集成,但这是我到目前为止所要做的:

    RedisMessageListenerContainer container = new RedisMessageListenerContainer() {
        @Override
        protected void handleSubscriptionException(Throwable ex) {
            super.handleSubscriptionException(ex); // don't know what actually happened in here and no way to find out :/
            if (ex instanceof RedisConnectionFailureException) {
                // handled by super hopefully, don't care
            } else if (ex instanceof InterruptedException){
                // can ignore those I guess
            } else if (ex instanceof TransientDataAccessException || ex instanceof RecoverableDataAccessException) {
                // try to restart in those cases?
                if (isRunning()) {
                    logger.error("Connection failure occurred. Restarting subscription task manually due to " + ex, ex);
                    sleepBeforeRecoveryAttempt();
                    start(); // best we can do
                }
            } else {
                // otherwise shutdown and hope for the best next time
                if (isRunning()) {
                    logger.warn("Shutting down application due to unknown exception " + ex, ex);
                    context.close();
                }
            }
        }
    };

0 个答案:

没有答案