[AMQNET-488] Deadlock in InactivityMonitor when using Failover

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.6.2, 1.6.3
• Fix Version/s: 1.6.4, 1.7.0
• Component/s: ActiveMQ

Environment

ActiveMQ 5.10.0 on Windows Server 2012
Apache.NMS.ActiveMQ 6.2 on .Net 4.5 + Windows 7

• Patch Info: Patch Available
• Assignee: Timothy A. Bish
• Reporter: Laurence Pike
• Created: 12/Aug/14 10:06
• Updated: 12/Aug/14 17:45
• Resolved: 12/Aug/14 17:45

Description

Can occur when FailoverTransport is disposing the transport when the inactivity monitor fires. Deadlock between InactivityMonitor.monitor and CompositeTaskRunner.mutex.

Stack traces for deadlock:

mscorlib.dll!System.Threading.Monitor.Enter(object obj, ref bool lockTaken) Unknown
Apache.NMS.ActiveMQ.dll!Apache.NMS.ActiveMQ.Transport.InactivityMonitor.StopMonitorThreads() Line 381 C#
Apache.NMS.ActiveMQ.dll!Apache.NMS.ActiveMQ.Transport.InactivityMonitor.OnException(Apache.NMS.ActiveMQ.Transport.ITransport sender, System.Exception command) Line 310 C#
Apache.NMS.ActiveMQ.dll!Apache.NMS.ActiveMQ.Transport.Tcp.TcpTransport.Oneway(Apache.NMS.ActiveMQ.Commands.Command command) Line 146 C#
Apache.NMS.ActiveMQ.dll!Apache.NMS.ActiveMQ.Transport.InactivityMonitor.Oneway(Apache.NMS.ActiveMQ.Commands.Command command) Line 294 C#
Apache.NMS.ActiveMQ.dll!Apache.NMS.ActiveMQ.Transport.InactivityMonitor.AsyncWriteTask.Iterate() Line 474 C#
Apache.NMS.ActiveMQ.dll!Apache.NMS.ActiveMQ.Threads.CompositeTaskRunner.Iterate() Line 194 C#
Apache.NMS.ActiveMQ.dll!Apache.NMS.ActiveMQ.Threads.CompositeTaskRunner.Run() Line 149 C#
mscorlib.dll!System.Threading.ThreadHelper.ThreadStart_Context(object state) Unknown
mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Unknown
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Unknown
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Unknown
mscorlib.dll!System.Threading.ThreadHelper.ThreadStart() Unknown

============================================================================================================================

mscorlib.dll!System.Threading.Monitor.Enter(object obj, ref bool lockTaken) Unknown
Apache.NMS.ActiveMQ.dll!Apache.NMS.ActiveMQ.Threads.CompositeTaskRunner.RemoveTask(Apache.NMS.ActiveMQ.Threads.CompositeTask task) Line 65 C#
Apache.NMS.ActiveMQ.dll!Apache.NMS.ActiveMQ.Transport.InactivityMonitor.StopMonitorThreads() Line 401 C#
Apache.NMS.ActiveMQ.dll!Apache.NMS.ActiveMQ.Transport.InactivityMonitor.Stop() Line 223 C#
Apache.NMS.ActiveMQ.dll!Apache.NMS.ActiveMQ.Transport.TransportFilter.Stop() Line 196 C#
Apache.NMS.ActiveMQ.dll!Apache.NMS.ActiveMQ.Transport.Failover.FailoverTransport.DisposeTransport(Apache.NMS.ActiveMQ.Transport.ITransport transport) Line 1521 C#
Apache.NMS.ActiveMQ.dll!Apache.NMS.ActiveMQ.Transport.Failover.FailoverTransport.HandleTransportFailure(System.Exception e) Line 454 C#
Apache.NMS.ActiveMQ.dll!Apache.NMS.ActiveMQ.Transport.Failover.FailoverTransport.Oneway(Apache.NMS.ActiveMQ.Commands.Command command) Line 815 C#
Apache.NMS.ActiveMQ.dll!Apache.NMS.ActiveMQ.Transport.TransportFilter.Oneway(Apache.NMS.ActiveMQ.Commands.Command command) Line 82 C#
Apache.NMS.ActiveMQ.dll!Apache.NMS.ActiveMQ.Transport.MutexTransport.Oneway(Apache.NMS.ActiveMQ.Commands.Command command) Line 69 C#
Apache.NMS.ActiveMQ.dll!Apache.NMS.ActiveMQ.Transport.ResponseCorrelator.Oneway(Apache.NMS.ActiveMQ.Commands.Command command) Line 54 C#
Apache.NMS.ActiveMQ.dll!Apache.NMS.ActiveMQ.Connection.Oneway(Apache.NMS.ActiveMQ.Commands.Command command) Line 883 C#
Apache.NMS.ActiveMQ.dll!Apache.NMS.ActiveMQ.Session.Session(Apache.NMS.ActiveMQ.Connection connection, Apache.NMS.ActiveMQ.Commands.SessionId sessionId, Apache.NMS.AcknowledgementMode acknowledgementMode) Line 72 C#
Apache.NMS.ActiveMQ.dll!Apache.NMS.ActiveMQ.Connection.CreateActiveMQSession(Apache.NMS.AcknowledgementMode ackMode) Line 594 C#
Apache.NMS.ActiveMQ.dll!Apache.NMS.ActiveMQ.Connection.CreateSession(Apache.NMS.AcknowledgementMode sessionAcknowledgementMode) Line 588 C#
[Private code sending message in new session]