Qpid
  1. Qpid
  2. QPID-2045

crash when destroying a federation link with a dynamic bridge

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.6
    • Component/s: C++ Broker
    • Labels:
      None
    • Environment:

      fedora core 11

      Description

      While experimenting with the Federation tests, I managed to cause a crash in the broker. My modified test creates two bridges, but only calls "bridge.close()" on one of the bridges. This results in a broker crash. I will attach my test shortly. I am running the latest trunk as of today (8/11/2009). Thanks,

      -K


      pure virtual method called
      terminate called without an active exception

      Program received signal SIGABRT, Aborted.
      [Switching to Thread 0x7ffff5001910 (LWP 8354)]
      0x00007ffff65032f5 in raise () from /lib64/libc.so.6
      Missing separate debuginfos, use: debuginfo-install boost-1.37.0-6.fc11.x86_64 cyrus-sasl-lib-2.1.22-22.fc11.x86_64 e2fsprogs-libs-1.41.4-12.fc11.x86_64 glibc-2.10.1-4.x86_64 libgcc-4.4.0-4.x86_64 libstdc++-4.4.0-4.x86_64 nss-softokn-freebl-3.12.3.99.3-2.11.3.fc11.x86_64
      (gdb) where
      warning: (Internal error: pc 0x7ffff76e55de in read in psymtab, but not in symtab.)

      warning: (Internal error: pc 0x7ffff76e55de in read in psymtab, but not in symtab.)

      warning: (Internal error: pc 0x7ffff76e5590 in read in psymtab, but not in symtab.)

      #0 0x00007ffff65032f5 in raise () from /lib64/libc.so.6
      #1 0x00007ffff6504b20 in abort () from /lib64/libc.so.6
      #2 0x0000003d972c3e15 in _gnu_cxx::_verbose_terminate_handler() () from /usr/lib64/libstdc++.so.6
      #3 0x0000003d972c2236 in ?? () from /usr/lib64/libstdc++.so.6
      #4 0x0000003d972c2263 in std::terminate() () from /usr/lib64/libstdc++.so.6
      #5 0x0000003d972c2b3f in __cxa_pure_virtual () from /usr/lib64/libstdc++.so.6
      #6 0x00007ffff7abd565 in qpid::broker::Exchange::propagateFedOp (this=0x7ffff0034620, routingKey="ft-key.one.#", tags="", op="U", origin="") at qpid/broker/Exchange.cpp:237
      #7 0x00007ffff7b34975 in qpid::broker::TopicExchange::unbind (this=<value optimized out>, queue=<value optimized out>, constRoutingKey=<value optimized out>) at qpid/broker/TopicExchange.cpp:267
      #8 0x00007ffff7af6911 in qpid::broker::QueueBindings::unbind (this=0x7ffff0040058, exchanges=<value optimized out>, queue={px = 0x7ffff003fe10, pn = {pi_ = 0x7ffff003b370}}) at qpid/broker/QueueBindings.cpp:39
      #9 0x00007ffff7aed1be in qpid::broker::Queue::unbind (this=<value optimized out>, exchanges=@0x20a2, shared_ref=<value optimized out>) at qpid/broker/Queue.cpp:849
      #10 0x00007ffff7aed390 in qpid::broker::Queue::tryAutoDelete (broker=@0x629bb0, queue={px = 0x7ffff003fe10, pn = {pi_ = 0x7ffff003b370}}) at qpid/broker/Queue.cpp:927
      #11 0x00007ffff7b0e728 in qpid::broker::SemanticState::cancel (this=<value optimized out>, c=<value optimized out>) at qpid/broker/SemanticState.cpp:342
      #12 0x00007ffff7b12726 in qpid::broker::SemanticState::~SemanticState (this=0x7ffff00340c8, __in_chrg=<value optimized out>) at qpid/broker/SemanticState.cpp:73
      #13 0x00007ffff7b2f858 in qpid::broker::SessionState::~SessionState (this=0x7ffff0033ef0, __in_chrg=<value optimized out>) at qpid/broker/SessionState.cpp:96
      #14 0x00007ffff7b2825a in ~auto_ptr (this=<value optimized out>, __in_chrg=<value optimized out>) at /usr/lib/gcc/x86_64-redhat-linux/4.4.0/../../../../include/c++/4.4.0/backward/auto_ptr.h:168
      #15 qpid::broker::SessionHandler::handleDetach (this=<value optimized out>, __in_chrg=<value optimized out>) at qpid/broker/SessionHandler.cpp:70
      #16 0x00007ffff76e478a in qpid::amqp_0_10::SessionHandler::detach (this=0x7ffff0000b40, name="test-session") at qpid/amqp_0_10/SessionHandler.cpp:176
      #17 0x00007ffff76a5e33 in qpid::framing::SessionDetachBody::invoke<qpid::framing::AMQP_AllOperations::SessionHandler> (invocable=<value optimized out>, this=<value optimized out>) at ../include/qpid/framing/SessionDetachBody.h:63
      #18 qpid::framing::AMQP_AllOperations::SessionHandler::Invoker::visit (invocable=<value optimized out>, this=<value optimized out>) at qpid/framing/AllInvoker.cpp:744
      #19 0x00007ffff76e55df in qpid::framing::invoke<qpid::amqp_0_10::SessionHandler> (target=<value optimized out>, body=@0x6) at ./qpid/framing/Invoker.h:67
      #20 0x00007ffff76e1765 in qpid::amqp_0_10::SessionHandler::invoke (this=0x0, m=@0x6) at qpid/amqp_0_10/SessionHandler.cpp:72
      #21 0x00007ffff76e2c00 in qpid::amqp_0_10::SessionHandler::handleIn (this=0x7ffff0000b40, f=@0x7ffff50004d0) at qpid/amqp_0_10/SessionHandler.cpp:82
      #22 0x00007ffff7a9ab23 in qpid::framing::Handler<qpid::framing::AMQFrame&>::operator() (t=<value optimized out>, this=<value optimized out>) at ./qpid/framing/Handler.h:42
      #23 qpid::broker::Connection::received (t=<value optimized out>, this=<value optimized out>) at qpid/broker/Connection.cpp:136
      #24 0x00007ffff7a7e8e8 in qpid::amqp_0_10::Connection::decode (this=0x7ffff00012a0, buffer=<value optimized out>, size=<value optimized out>) at qpid/amqp_0_10/Connection.cpp:58
      #25 0x00007ffff770c0b4 in qpid::sys::AsynchIOHandler::readbuff (this=0x7ffff00356d0, buff=0x7ffff0043530) at qpid/sys/AsynchIOHandler.cpp:113
      #26 0x00007ffff7657c44 in boost::function2<bool, qpid::sys::AsynchIO&, qpid::sys::AsynchIOBufferBase*>::operator() (this=<value optimized out>, a0=@0x20a2, a1=0x6) at /usr/include/boost/function/function_template.hpp:989
      #27 0x00007ffff7655e29 in qpid::sys::posix::AsynchIO::readable (this=0x7ffff0035d60, h=<value optimized out>) at qpid/sys/posix/AsynchIO.cpp:452
      #28 0x00007ffff7712b74 in boost::function1<void, qpid::sys::DispatchHandle&>::operator() (this=<value optimized out>, a0=@0x20a2) at /usr/include/boost/function/function_template.hpp:989
      #29 0x00007ffff770f8ac in qpid::sys::DispatchHandle::processEvent (this=0x7ffff0035d68, type=qpid::sys::Poller::READABLE) at qpid/sys/DispatchHandle.cpp:278
      #30 0x00007ffff766316a in qpid::sys::Poller::Event::process (this=<value optimized out>) at ./qpid/sys/Poller.h:123
      #31 qpid::sys::Poller::run (this=<value optimized out>) at qpid/sys/epoll/EpollPoller.cpp:476
      #32 0x00007ffff7659baa in qpid::sys::(anonymous namespace)::runRunnable (p=0x209e) at qpid/sys/posix/Thread.cpp:35
      #33 0x00007ffff62ba86a in start_thread () from /lib64/libpthread.so.0
      #34 0x00007ffff65ae39d in clone () from /lib64/libc.so.6
      #35 0x0000000000000000 in ?? ()
      (gdb)

      1. federation.patch
        4 kB
        Ken Giusti
      2. QPID-2045.diff
        0.7 kB
        Ken Giusti
      3. QPID-2045_fix.patch
        0.7 kB
        Ken Giusti
      4. QPID-2045_test.patch
        0.8 kB
        Ken Giusti

        Activity

        Hide
        Ken Giusti added a comment -

        diff against qpid/cpp/src/tests/federation.py test script - includes my test which causes the crash.

        Show
        Ken Giusti added a comment - diff against qpid/cpp/src/tests/federation.py test script - includes my test which causes the crash.
        Hide
        Ken Giusti added a comment -

        Potential cause:

        The method Link::destroy() removes bridges from the "active" vector member, but does not invoke the bridge's ::closed() method.

        Other Link class methods that clean the active vector - Link::closed() and Link::cancel() - manually invoke the bridge's ::closed() method for every bridge that is removed from the active list.

        I tried adding a call to the bridge's closed() method from within Link::destroy() active vector cleanup - no crash. This seems correct in the fact that the bridge's create() method (which is cancelled by the bridge's closed() method) is invoked by Link when the bridge is added to the active list.

        Thoughts?

        -K

        Show
        Ken Giusti added a comment - Potential cause: The method Link::destroy() removes bridges from the "active" vector member, but does not invoke the bridge's ::closed() method. Other Link class methods that clean the active vector - Link::closed() and Link::cancel() - manually invoke the bridge's ::closed() method for every bridge that is removed from the active list. I tried adding a call to the bridge's closed() method from within Link::destroy() active vector cleanup - no crash. This seems correct in the fact that the bridge's create() method (which is cancelled by the bridge's closed() method) is invoked by Link when the bridge is added to the active list. Thoughts? -K
        Hide
        Ken Giusti added a comment -

        Proposed fix: call bridge's closed() method when removing it from the active vector.

        Show
        Ken Giusti added a comment - Proposed fix: call bridge's closed() method when removing it from the active vector.
        Hide
        Ken Giusti added a comment -

        Cleaned up the federation test patch, and retested.

        Show
        Ken Giusti added a comment - Cleaned up the federation test patch, and retested.
        Hide
        Ted Ross added a comment -

        Committed Ken's test and fix patches.

        Show
        Ted Ross added a comment - Committed Ken's test and fix patches.

          People

          • Assignee:
            Ted Ross
            Reporter:
            Ken Giusti
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development