Uploaded image for project: 'ActiveMQ .Net'
  1. ActiveMQ .Net
  2. AMQNET-366

Deadlock when TCP connection breaks



    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.5.2
    • 1.5.3, 1.6.0
    • ActiveMQ
    • None


      When an ActiveMQ TCP connection gets broken (for instance when the ActiveMQ server goes down), the AsyncCallExceptionListener is used by helper classes like SingleConnectionFactory and SimpleMessageListenerContainer from springframework.net to reconnect when connection becomes available again.
      From this thread, these classes will call Close() on the connection which in turn calls Shutdown on the ThreadPoolExecutor (from the final block in Connection.cs:Close()).
      Inside Shutdown(), the execution hangs on this.executionComplete.WaitOne(), and there is a deadlock because the AsyncCallExceptionListener call never returns, and nobody else triggers the executionComplete event

      Simple test case to reproduce the problem:

      using System;
      using System.Threading;
      using Apache.NMS.ActiveMQ;
      using NUnit.Framework;
      using Spring.Messaging.Nms.Listener;

      public class ActiveMQTests
      public void TestConnection()
      var connectionFactory = new ConnectionFactory("tcp://localhost:61616")

      {UserName = "qwe", Password = "asd"}


      var messageListenerContainer = new SimpleMessageListenerContainer

      { ConnectionFactory = connectionFactory, DestinationName = "test.test.in" }




      Start debugging. Wait until it connects tothe ActiveMQ server, shutdown the ActiveMQ server, pause execution and look at the Threads.

      Example call stack:

      [In a sleep, wait, or join]
      [External Code]
      Apache.NMS.ActiveMQ.DLL!Apache.NMS.ActiveMQ.Threads.ThreadPoolExecutor.Shutdown() Line 121 + 0xe bytes
      Apache.NMS.ActiveMQ.DLL!Apache.NMS.ActiveMQ.Connection.Close() Line 592 + 0xd bytes
      Spring.Messaging.Nms.DLL!Spring.Messaging.Nms.Connections.ConnectionFactoryUtils.ReleaseConnection(Apache.NMS.IConnection connection, Apache.NMS.IConnectionFactory cf, bool started) Line 74 + 0x9 bytes
      Spring.Messaging.Nms.DLL!Spring.Messaging.Nms.Listener.AbstractListenerContainer.RefreshSharedConnection() Line 432
      Spring.Messaging.Nms.DLL!Spring.Messaging.Nms.Listener.SimpleMessageListenerContainer.RefreshConnectionUntilSuccessful() Line 242
      Spring.Messaging.Nms.DLL!Spring.Messaging.Nms.Listener.SimpleMessageListenerContainer.OnException(System.Exception exception) Line 210
      Apache.NMS.ActiveMQ.DLL!Apache.NMS.ActiveMQ.Connection.AsyncCallExceptionListener(object error) Line 945 + 0x11 bytes
      Apache.NMS.ActiveMQ.DLL!Apache.NMS.ActiveMQ.Threads.ThreadPoolExecutor.Future.Run() Line 63 + 0x14 bytes
      Apache.NMS.ActiveMQ.DLL!Apache.NMS.ActiveMQ.Threads.ThreadPoolExecutor.QueueProcessor(object unused) Line 150 + 0xa bytes
      [External Code]

      Should it not be possible to Close the connection from the AsyncCallExceptionListener thread?

      I've tried running executor.Shutdown() in Connection.Close (under finally block) in a new thread and recompiling the NMS.ActiveMQ DLL, Like so:

      if(executor != null)
      var thread = new Thread(delegate()

      { executor.Shutdown(); }


      This seems to solve the problem, however I am not certain that this will not break anything else.

      I am using Spring framework.net version 1.3.2 and apache.NMS version 1.5.0


        1. ActiveMQTestWithoutSpring.cs
          2 kB
          Mikael Finstad
        2. ActiveMQTests.cs
          0.6 kB
          Mikael Finstad



            tabish Timothy A. Bish
            mifi Mikael Finstad
            0 Vote for this issue
            0 Start watching this issue