当我尝试释放redis锁时,我一直收到以下超时异常。我正在使用Azure Redis和StackExchange.Redis版本1.1.603。我得到的例外情况如下。我查看了异常错误中包含的链接,但我没有看到任何与此方案有关的内容。
Unexpected lock release failure on lock key 1ea47191-85e6-4f89-ad20-be700d643d6a-ProbeIdsLock on machine ELITEBOOK. Timeout performing EX
EC, inst: 0, mgr: Inactive, err: never, queue: 3, qu: 1, qs: 2, qc: 0, wr: 1, wq: 1, in: 101, ar: 0, clientName: ELITEBOOK, IOCP: (Busy=1,Free=
999,Min=4,Max=1000), WORKER: (Busy=1,Free=1022,Min=4,Max=1023), Local-CPU: 100% (Please take a look at this article for some common client-side
issues that can cause timeouts: https://github.com/StackExchange/StackExchange.Redis/tree/master/Docs/Timeouts.md) - at StackExchange.Redis
.ConnectionMultiplexer.ExecuteSyncImpl[T](Message message, ResultProcessor`1 processor, ServerEndPoint server) in C:\TeamCity\buildAgent\work\3
ae0647004edff78\StackExchange.Redis\StackExchange\Redis\ConnectionMultiplexer.cs:line 2044
at StackExchange.Redis.RedisTransaction.Execute(CommandFlags flags) in C:\TeamCity\buildAgent\work\3ae0647004edff78\StackExchange.Redi
s\StackExchange\Redis\RedisTransaction.cs:line 51
at StackExchange.Redis.RedisDatabase.LockRelease(RedisKey key, RedisValue value, CommandFlags flags) in C:\TeamCity\buildAgent\work\3a
e0647004edff78\StackExchange.Redis\StackExchange\Redis\RedisDatabase.cs:line 838
at ENow.RedisCache.RedisConnection.LockRelease(RedisKey key, RedisValue value, CommandFlags commandFlags)
at ENow.RedisCache.MonitoringCache.LockRelease(RedisKey lockKey, RedisValue token)
代码基本上如下所示。 Redis调用被包装,以便最终编写单元测试更容易。
public async Task<bool> AddProbeStatus(string tenantId, ProbeStatus probeStatus)
{
probeStatus.ProbeId.ThrowIfNullOrEmpty("probeStatus.ProbeId");
var key = $"{tenantId}-ProbeIds";
var lockKey = key + "Lock";
RedisValue token = Environment.MachineName;
var lockAquired = false;
var finalResult = false;
var retryCount = 0;
try
{
while (!lockAquired)
{
retryCount++;
lockAquired = _redisConnection.LockTake(lockKey, token, TimeSpan.MaxValue);
if (lockAquired)
{
_log.LogDebug($"Probe status lock aquired for tenant {tenantId}");
var result = await _redisConnection.StringGetAsync(key);
List<ProbeStatus> probeStatuses;
if (result == RedisValue.Null)
{
_log.LogDebug(
$"No probe status cache found for tenant {tenantId}. Adding probe {probeStatus.ProbeId} to cache.");
probeStatus.LastCheckinTimeInUtc = DateTime.UtcNow;
probeStatuses = new List<ProbeStatus> {probeStatus};
}
else
{
probeStatuses = JsonConvert.DeserializeObject<List<ProbeStatus>>(result);
var existingProbeStatus = probeStatuses.FirstOrDefault(p => p.ProbeId == probeStatus.ProbeId);
if (existingProbeStatus != null)
{
_log.LogDebug(
$"Updating last checkin time for probe {probeStatus.ProbeId} for tenant {tenantId} in probe status cache.");
existingProbeStatus.LastCheckinTimeInUtc = probeStatus.LastCheckinTimeInUtc;
}
else
{
_log.LogDebug(
$"Adding new probe {probeStatus.ProbeId} for tenant {tenantId} to probe status cache.");
probeStatuses.Add(probeStatus);
}
}
var probeStatusesAsJson = JsonConvert.SerializeObject(probeStatuses);
finalResult = await _redisConnection.StringSetAsync(key, probeStatusesAsJson);
}
else
{
if (retryCount > MaxRetryCount)
{
var errorMessage =
$"Retry count exceeded while attempting to aquire lock in probe status cache for tenant {tenantId}";
_log.LogError(errorMessage);
throw new Exception(errorMessage);
}
_log.LogWarning($"Couldn't aquire lock in probe status cache for tenant {tenantId}. Waiting {DefaultRetrySeconds} seconds before retrying");
await Task.Delay(TimeSpan.FromSeconds(DefaultRetrySeconds));
}
}
}
catch (Exception ex)
{
_log.LogError($"Failed to set probe IDs for tenant {tenantId}. {ex.Message}");
throw;
}
finally
{
if (lockAquired)
{
if(!LockRelease(lockKey, token))
{
finalResult = false;
}
}
else
{
_log.LogDebug($"Lock not aquired for key {lockKey} on machine {token}");
}
}
return finalResult;
}
此代码是从ASP.NET 5 RC1 WebAPI调用的。如果我在Azure或本地运行它,我会得到超时。但是,我有一个简单的XUnit集成测试,它可以运行相同的代码,并且在同一个Azure托管的Redis缓存中无异常运行。是否存在与ASP.NET 5 RC1和StackExchange.Redis版本1.1.603不兼容的问题?奇怪的是,这个问题在WebAPI中始终存在。它不是任何间歇性的。此外,它总是成功使用集成测试,所以感觉就像其他东西在这里,并且超时只是一个红色的鲱鱼。