ActiveMQ C++ Client
  1. ActiveMQ C++ Client
  2. AMQCPP-520

AMQ crashes when an exception comes during the connection closure

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 3.6.0
    • Fix Version/s: 3.8.2, 3.9.0
    • Component/s: None
    • Environment:

      Solaris 10 with gcc 3.4.6

    • Patch Info:
      Patch Available
    • Regression:
      Regression

      Description

      From one thread, an exception comes and FailoverTransport::handleTransportFailure () is called:

      activemq::transport::failover::CloseTransportsTask::add(const Pointer<Transport> transport)
      activemq::transport::failover::FailoverTransport::handleTransportFailure(const decaf::lang::Exception & error=

      {...})
      activemq::transport::failover::FailoverTransportListener::onException(const decaf::lang::Exception & ex={...}

      )
      activemq::transport::TransportFilter::onException(const decaf::lang::Exception & ex=

      {...})
      activemq::wireformat::openwire::OpenWireFormatNegotiator::onException(const decaf::lang::Exception & ex={...}

      )
      activemq::transport::TransportFilter::onException(const decaf::lang::Exception & ex=

      {...})
      activemq::transport::inactivity::InactivityMonitor::onException(const decaf::lang::Exception & ex={...}

      )
      activemq::transport::TransportFilter::onException(const decaf::lang::Exception & ex=

      {...})
      activemq::transport::IOTransport::fire(decaf::lang::Exception & ex={...}

      )
      activemq::transport::IOTransport::run()
      decaf::lang::Thread::run()

      From another thread, the amq connection is being closed. It destructs the FailoverTransport without waiting that the first thread finishes to call handleTransportFailure():

      activemq::transport::failover::FailoverTransport::~FailoverTransport()
      activemq::transport::failover::FailoverTransport::`vector deleting destructor'(unsigned int)
      decaf::lang::Pointer<activemq::transport::Transport,decaf::util::concurrent::atomic::AtomicRefCounter>::onDeleteFunc(activemq::transport::Transport * value=0x02f307b0)
      decaf::lang::Pointer<activemq::transport::Transport,decaf::util::concurrent::atomic::AtomicRefCounter>::~Pointer<activemq::transport::Transport,decaf::util::concurrent::atomic::AtomicRefCounter>()
      decaf::lang::Pointer<activemq::transport::Transport,decaf::util::concurrent::atomic::AtomicRefCounter>::reset(activemq::transport::Transport * value=0x00000000)
      activemq::transport::TransportFilter::~TransportFilter()
      activemq::transport::correlator::ResponseCorrelator::~ResponseCorrelator()
      activemq::transport::correlator::ResponseCorrelator::`vector deleting destructor'(unsigned int)
      decaf::lang::Pointer<activemq::transport::Transport,decaf::util::concurrent::atomic::AtomicRefCounter>::onDeleteFunc(activemq::transport::Transport * value=0x03ba6058)
      decaf::lang::Pointer<activemq::transport::Transport,decaf::util::concurrent::atomic::AtomicRefCounter>::~Pointer<activemq::transport::Transport,decaf::util::concurrent::atomic::AtomicRefCounter>()
      decaf::lang::Pointer<activemq::transport::Transport,decaf::util::concurrent::atomic::AtomicRefCounter>::reset(activemq::transport::Transport * value=0x00000000)
      activemq::core::ActiveMQConnection::disconnect(__int64 lastDeliveredSequenceId=0)
      activemq::core::ActiveMQConnection::close()

      I don't have any simple test case as it is a race condition.
      I've attached a proposal patch.

      1. test.cc
        3 kB
        Daniel Laügt
      2. FailoverTransport.cpp.patch
        1 kB
        Daniel Laügt

        Activity

        Hide
        Timothy Bish added a comment -

        Fixed on trunk and 3.8.x

        Show
        Timothy Bish added a comment - Fixed on trunk and 3.8.x
        Hide
        Daniel Laügt added a comment -

        Probably two domains can be fixed:
        1) Before AMQCPP-463, FailoverTransport::handleTransportFailure() was multi-thread safe. It is not the case after the changes done in AMQCPP-463.
        2) An exception is raised on reading data from IOTransport. The exception "Not supported for SSL Sockets" is raised by OpenSSLSocket::shutdownInput(). This method is called by OpenSSLSocket::read() when the read status is SSL_ERROR_ZERO_RETURN and the socket is not closed.

        Here is OpenSLL documentation about SSL_ERROR_ZERO_RETURN:
        The TLS/SSL connection has been closed. If the protocol version is SSL 3.0 or TLS 1.0, this result code is returned only if a closure alert has occurred in the protocol, i.e. if the connection has been closed cleanly. Note that in this case SSL_ERROR_ZERO_RETURN does not necessarily indicate that the underlying transport has been closed.

        Show
        Daniel Laügt added a comment - Probably two domains can be fixed: 1) Before AMQCPP-463 , FailoverTransport::handleTransportFailure() was multi-thread safe. It is not the case after the changes done in AMQCPP-463 . 2) An exception is raised on reading data from IOTransport. The exception "Not supported for SSL Sockets" is raised by OpenSSLSocket::shutdownInput(). This method is called by OpenSSLSocket::read() when the read status is SSL_ERROR_ZERO_RETURN and the socket is not closed. Here is OpenSLL documentation about SSL_ERROR_ZERO_RETURN: The TLS/SSL connection has been closed. If the protocol version is SSL 3.0 or TLS 1.0, this result code is returned only if a closure alert has occurred in the protocol, i.e. if the connection has been closed cleanly. Note that in this case SSL_ERROR_ZERO_RETURN does not necessarily indicate that the underlying transport has been closed.
        Hide
        Daniel Laügt added a comment -

        I've reproduced it in 3.8.1. This occurs with OpenSSL socket.

        Test case:
        Run in parallel:

        • test client1
        • test client2
        • test client3
        • test client4
        Show
        Daniel Laügt added a comment - I've reproduced it in 3.8.1. This occurs with OpenSSL socket. Test case: Run in parallel: test client1 test client2 test client3 test client4
        Hide
        Timothy Bish added a comment -

        First thing to do is update to the latest release, v3.8.1. Then try and create a test case that shows the issue so it can be tested and a fix verified.

        Show
        Timothy Bish added a comment - First thing to do is update to the latest release, v3.8.1. Then try and create a test case that shows the issue so it can be tested and a fix verified.

          People

          • Assignee:
            Timothy Bish
            Reporter:
            Daniel Laügt
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 48h
              48h
              Remaining:
              Remaining Estimate - 48h
              48h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development