Redis超时以及如何分析Redis Timeout消息

时间:2017-07-19 08:09:34

标签: asp.net redis stackexchange.redis

我正在使用Redis开发一个用于缓存的asp.net MVC .NET 4.5.1应用程序。

  • 服务器:MSOpenTech Redis 3.0
  • 客户:StackExchange.Redis 1.2.4

在制作中,我有时会遇到Redis Timeout错误。我使用documentation尝试解决此问题。所以我们在Application_Start中设置了Threadpool限制:

ThreadPool.SetMinThreads(200, 200)

从那以后我们没有多少时间,但偶尔也会发生。但我接受了。

但现在我注意到在我的开发机器上运行web& Redis on localhost。

Redis客户端设置

    private static readonly Lazy<ConnectionMultiplexer> LazyConnection = new Lazy<ConnectionMultiplexer>(() =>
    {
        IRedisConfiguration redisConfiguration = new RedisWebConfigConfiguration();

        var config = new ConfigurationOptions();
        config.EndPoints.Add(redisConfiguration.HostName);
        config.Ssl = false;
        config.AbortOnConnectFail = false;
        config.ConnectRetry = 10;
        config.ConnectTimeout = 5000;
        config.SyncTimeout = 5000;

        var connectionMultiplexer = ConnectionMultiplexer.Connect(config);

        connectionMultiplexer.ConfigurationChanged += _redisConnection_ConfigurationChanged;
        connectionMultiplexer.ConnectionFailed += _redisConnection_ConnectionFailed;
        connectionMultiplexer.ConnectionRestored += RedisConnectionOnConnectionRestored;
        connectionMultiplexer.ErrorMessage += _redisConnection_ErrorMessage;
        connectionMultiplexer.InternalError += _redisConnection_InternalError;

        return connectionMultiplexer;
    });

例外:

StackExchange.Redis.RedisTimeoutException: 'Timeout performing EXISTS 
Supplier:1:GroupIds, inst: 1, queue: 14, qu: 0, qs: 14, qc: 0, wr: 0, wq: 
0, in: 388, ar: 0, clientName: WINTERFELL, serverEndpoint: 127.0.0.1:6379, 
keyHashSlot: 394, IOCP: (Busy=1,Free=999,Min=200,Max=1000), WORKER: 
(Busy=2,Free=32765,Min=200,Max=32767) (Please take a look at this article for 
some common client-side issues that can cause timeouts: 
http://stackexchange.github.io/StackExchange.Redis/Timeouts)'

Redis监控de CLI的输出:

1500450694.235873 [0 127.0.0.1:55017] "GET" "user:9920:profileimage"
1500450694.633475 [0 127.0.0.1:55017] "PING"
1500450695.641077 [0 127.0.0.1:55017] "PING"
1500450696.642158 [0 127.0.0.1:55017] "PING"
1500450697.643100 [0 127.0.0.1:55017] "PING"
1500450698.653751 [0 127.0.0.1:55017] "PING"
1500450699.623142 [0 127.0.0.1:55017] "EXISTS" "Supplier:1:GroupIds"
1500450699.654927 [0 127.0.0.1:55017] "PING"
1500450699.986743 [0 127.0.0.1:55017] "EXISTS" "Supplier:1:GroupIds"
1500450700.297318 [0 127.0.0.1:55017] "GET" "user:9920:profileimage"
1500450700.655792 [0 127.0.0.1:55017] "PING"
1500450701.656635 [0 127.0.0.1:55017] "PING"
1500450702.659291 [0 127.0.0.1:55017] "PING"
1500450703.672797 [0 127.0.0.1:55017] "PING"

我认为这是一种奇怪的行为,因为这一切都在localhost上,没有任何负载或延迟。 我应该从哪里开始调查此错误?它是Redis服务器端问题还是StackExchange客户端问题。或者也许是网络/端口?

欢迎任何建议。非常感谢提前。

编辑:添加事件处理程序

    private static void RedisConnectionOnConnectionRestored(object sender, ConnectionFailedEventArgs e)
    {
        if (e == null)
        {
            CurrentLogger.Logger.Debug("RedisConnectionOnConnectionRestored - ConnectionFailedEventArgs is null");
            return;
        }

        try
        {

            string connectionType = e.ConnectionType.ToString();
            string endPoint = e.EndPoint?.ToString() ?? "";
            string exception = e.Exception != null ? LogUtility.BuildExceptionMessage(e.Exception) : "";
            string failureType = e.FailureType.ToString();

            bool redisConnectionIsConnected = Connection.IsConnected;

            string debugMessage =
                $"REDIS ConnectionRestored. IsConnected: {redisConnectionIsConnected}. Connectiontype: {connectionType}. EndPoint: {endPoint}. Exception: {exception}. FailureType: {failureType}";

            CurrentLogger.Logger.Debug(debugMessage);


        }
        catch (Exception ex)
        {
            CurrentLogger.Logger.Error("Error writing DEBUG ConnectionRestored REDIS: " + ex.Message);
        }


    }

    private static void _redisConnection_ConnectionFailed(object sender, ConnectionFailedEventArgs e)
    {
        try
        {
            string errorMessage =
                string.Format(
                    "REDIS ConnectionFailed. Connectiontype: {0}. EndPoint: {1}. Exception: {2}. FailureType: {3}"
                    , e.ConnectionType.ToString()
                    , e.EndPoint
                    , LogUtility.BuildExceptionMessage(e.Exception)
                    , e.FailureType
                    );

            CurrentLogger.Logger.Error(errorMessage);
        }
        catch (Exception ex)
        {
            CurrentLogger.Logger.Error("Error writing DEBUG ConnectionFailed REDIS: " + ex.Message);
        }
    }

0 个答案:

没有答案