MassTransit 2.10.0导致LockRecursionException

时间:2016-02-19 06:22:29

标签: rabbitmq masstransit

我遇到了一个问题,在通过RabbitMq在MassTransit中发送消息时,偶尔会抛出LockRecursionException。

  

System.Threading.LockRecursionException:保持读锁定时可能无法获取写锁定。这种模式容易出现死锁

在类MassTransit.Transports.DefaultConnectionPolicy中引发异常,看起来Execute方法获取读锁定,运行回调并始终通过finally块释放读锁定。但是,无论出于何种原因,回调都会引发InvalidConnectionException,代码会尝试Reconnect()Reconnect()将尝试在TryEnterWriteLock中获取写锁定,并以System.Threading.LockRecursionException结束,因为当前线程已经保持读锁定。我没有看到,如果Finally块应该释放读锁定,那么可能会发生这种情况,除非ExitReadLock没有做它想做的事情。

我已经从MassTransit源代码和完整堆栈跟踪中包含了完整的类。

public class DefaultConnectionPolicy : ConnectionPolicy
{
    readonly ConnectionHandler _connectionHandler;
    readonly TimeSpan _reconnectDelay;
    readonly ILog _log = Logger.Get(typeof(DefaultConnectionPolicy));
    readonly ReaderWriterLockSlim _connectionLock = new ReaderWriterLockSlim(LockRecursionPolicy.SupportsRecursion);

    public DefaultConnectionPolicy(ConnectionHandler connectionHandler)
    {
        _connectionHandler = connectionHandler;
        _reconnectDelay = 10.Seconds();
    }

    public void Execute(Action callback)
    {
        try
        {
            try
            {
                // wait here so we can be sure that there is not a reconnect in progress
                _connectionLock.EnterReadLock();
                callback();
            }
            finally
            {
                _connectionLock.ExitReadLock();
            }
        }
        catch (InvalidConnectionException ex)
        {
            _log.Warn("Invalid Connection when executing callback", ex.InnerException);

            Reconnect();

            if (_log.IsDebugEnabled)
            {
                _log.Debug("Retrying callback after reconnect.");
            }

            try
            {
                // wait here so we can be sure that there is not a reconnect in progress
                _connectionLock.EnterReadLock();
                callback();
            }
            finally
            {
                _connectionLock.ExitReadLock();
            }
        }
    }

    void Reconnect()
    {
        if (_connectionLock.TryEnterWriteLock((int)_reconnectDelay.TotalMilliseconds/2))
        {
            try
            {
                if (_log.IsDebugEnabled)
                {
                    _log.Debug("Disconnecting connection handler.");
                }
                _connectionHandler.Disconnect();

                if (_reconnectDelay > TimeSpan.Zero)
                    Thread.Sleep(_reconnectDelay);

                if (_log.IsDebugEnabled)
                {
                    _log.Debug("Re-connecting connection handler...");
                }
                _connectionHandler.Connect();
            }
            catch (Exception)
            {
                _log.Warn("Failed to reconnect, deferring to connection policy for reconnection");
                _connectionHandler.ForceReconnect(_reconnectDelay);
            }
            finally
            {
                _connectionLock.ExitWriteLock();
            }
        }
        else
        {
            try
            {
                _connectionLock.EnterReadLock();
                if (_log.IsDebugEnabled)
                {
                    _log.Debug("Waiting for reconnect in another thread.");
                }
            }
            finally
            {
                _connectionLock.ExitReadLock();
            }
        }
    }
}

完整的堆栈跟踪

An exception was thrown during Send ---> System.Threading.LockRecursionException: Write lock may not be acquired with read lock held. This pattern is prone to deadlocks. Please ensure that read locks are released before taking a write lock. If an upgrade is necessary, use an upgrade lock in place of the read lock.
at System.Threading.ReaderWriterLockSlim.TryEnterWriteLockCore(TimeoutTracker timeout)
at System.Threading.ReaderWriterLockSlim.TryEnterWriteLock(TimeoutTracker timeout)
at MassTransit.Transports.DefaultConnectionPolicy.Reconnect() in z:\Builds\work\4ed32a1c3fc3f594\src\MassTransit\Transports\DefaultConnectionPolicy.cs:line 75
at MassTransit.Transports.DefaultConnectionPolicy.Execute(Action callback) in z:\Builds\work\4ed32a1c3fc3f594\src\MassTransit\Transports\DefaultConnectionPolicy.cs:line 53
at MassTransit.Transports.ConnectionPolicyChainImpl.Next(Action callback) in z:\Builds\work\4ed32a1c3fc3f594\src\MassTransit\Transports\ConnectionPolicyChainImpl.cs:line 49
at MassTransit.Transports.ConnectionHandlerImpl`1.Use(Action`1 callback) in z:\Builds\work\4ed32a1c3fc3f594\src\MassTransit\Transports\ConnectionHandlerImpl.cs:line 86
at MassTransit.Transports.RabbitMq.OutboundRabbitMqTransport.Send(ISendContext context) in z:\Builds\work\4ed32a1c3fc3f594\src\Transports\MassTransit.Transports.RabbitMq\OutboundRabbitMqTransport.cs:line 51
at MassTransit.Transports.Transport.Send(ISendContext context) in z:\Builds\work\4ed32a1c3fc3f594\src\MassTransit\Transports\Transport.cs:line 50
at MassTransit.Transports.Endpoint.Send[T](ISendContext`1 context) in z:\Builds\work\4ed32a1c3fc3f594\src\MassTransit\Transports\Endpoint.cs:line 111
--- End of inner exception stack trace ---
at MassTransit.Transports.Endpoint.Send[T](ISendContext`1 context) in z:\Builds\work\4ed32a1c3fc3f594\src\MassTransit\Transports\Endpoint.cs:line 117
at MassTransit.Pipeline.Sinks.EndpointMessageSink`1.<Enumerate>b__0(IBusPublishContext`1 x) in z:\Builds\work\4ed32a1c3fc3f594\src\MassTransit\Pipeline\Sinks\EndpointMessageSink.cs:line 45
at MassTransit.Pipeline.Sinks.OutboundConvertMessageSink`1.<>c__DisplayClass2.<>c__DisplayClass4.<Enumerate>b__1(ISendContext x) in z:\Builds\work\4ed32a1c3fc3f594\src\MassTransit\Pipeline\Sinks\OutboundConvertMessageSink.cs:line 36
at MassTransit.ServiceBus.Publish[T](T message, Action`1 contextCallback) in z:\Builds\work\4ed32a1c3fc3f594\src\MassTransit\ServiceBus.cs:line 180
--- End of inner exception stack trace ---
at MassTransit.ServiceBus.Publish[T](T message, Action`1 contextCallback) in z:\Builds\work\4ed32a1c3fc3f594\src\MassTransit\ServiceBus.cs:line 207
at MassTransit.Context.BusObjectPublisherImpl`1.Publish(IServiceBus bus, Object message, Action`1 contextCallback) in z:\Builds\work\4ed32a1c3fc3f594\src\MassTransit\Context\BusObjectPublisherImpl.cs:line 30
at MassTransit.ServiceBus.Publish(Object message, Type messageType, Action`1 contextCallback) in z:\Builds\work\4ed32a1c3fc3f594\src\MassTransit\ServiceBus.cs:line 244
at CollectionHouse.MassTransit.Messaging.MassTransitMessageBus.PublishImmediate(Object message)
at CollectionHouse.MassTransit.Services.MassTransitPublisher.Publish(Object message)
at CollectionHouse.MicroBus.EventBroadcast.MessageBroadcastHandler.<Handle>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Enexure.MicroBus.PipelineBuilder.<>c__DisplayClass7_0.<<GenerateNext>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at CollectionHouse.MicroBus.PipelineHandlers.LoggingHandler.<Handle>d__3.MoveNext()

0 个答案:

没有答案