Qpid
  1. Qpid
  2. QPID-4031

Deadlock when destroying a broker Link

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.16
    • Fix Version/s: 0.17
    • Component/s: C++ Broker
    • Labels:
      None

      Description

      Very unlikely, but if the Link's maintenance timer fires at the moment the Link is destroyed, a deadlock can occur.

        Activity

        Ken Giusti created issue -
        Hide
        Ken Giusti added a comment -

        Here's a backtrace of the lockup:

        Thread 3 (Thread 0x7f49b3bff700 (LWP 24447)):
        #0 0x00000032ab60e34d in __lll_lock_wait () from /lib64/libpthread.so.0
        #1 0x00000032ab609fb2 in _L_lock_910 () from /lib64/libpthread.so.0
        #2 0x00000032ab609e4b in pthread_mutex_lock () from /lib64/libpthread.so.0
        [attempts to take Link->lock]
        #3 0x00007f49b4b91b19 in qpid::broker::Link::maintenanceVisit() () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidbroker.so.2
        #4 0x00007f49b4b94db1 in qpid::broker::LinkTimerTask::fire() () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidbroker.so.2
        [take timer->callbackLock]
        #5 0x00007f49b46ecfe1 in qpid::sys::Timer::fire(boost::intrusive_ptr<qpid::sys::TimerTask>) () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidcommon.so.2
        #6 0x00007f49b46ee486 in qpid::sys::Timer::run() () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidcommon.so.2
        #7 0x00007f49b4611d4a in qpid::sys::(anonymous namespace)::runRunnable(void*) () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidcommon.so.2
        #8 0x00000032ab607d90 in start_thread () from /lib64/libpthread.so.0
        #9 0x00000032ab2f0f5d in clone () from /lib64/libc.so.6

        Thread 2 (Thread 0x7f49b337d700 (LWP 24448)):
        #0 0x00000032ab60e34d in __lll_lock_wait () from /lib64/libpthread.so.0
        #1 0x00000032ab609fb2 in _L_lock_910 () from /lib64/libpthread.so.0
        #2 0x00000032ab609e4b in pthread_mutex_lock () from /lib64/libpthread.so.0
        [attempts to take timer->callbackLock]
        #3 0x00007f49b46ed202 in qpid::sys::TimerTask::cancel() () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidcommon.so.2
        [takes link->lock]
        #4 0x00007f49b4b8f42e in qpid::broker::Link::destroy() () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidbroker.so.2
        #5 0x00007f49b4b4ebd8 in qpid::broker::Connection::doIoCallbacks() () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidbroker.so.2
        #6 0x00007f49b4b51526 in qpid::broker::Connection::doOutput() () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidbroker.so.2
        #7 0x00007f49b4b21a0d in qpid::amqp_0_10::Connection::canEncode() () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidbroker.so.2
        #8 0x00007f49b46e3b75 in qpid::sys::AsynchIOHandler::idle(qpid::sys::AsynchIO&) () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidcommon.so.2
        #9 0x00007f49b460f47d in qpid::sys::posix::AsynchIO::writeable(qpid::sys::DispatchHandle&) () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidcommon.so.2
        #10 0x00007f49b46e8796 in qpid::sys::DispatchHandle::processEvent(qpid::sys::Poller::EventType) () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidcommon.so.2
        #11 0x00007f49b461730f in qpid::sys::Poller::run() () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidcommon.so.2
        #12 0x00007f49b4611d4a in qpid::sys::(anonymous namespace)::runRunnable(void*) () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidcommon.so.2
        #13 0x00000032ab607d90 in start_thread () from /lib64/libpthread.so.0
        #14 0x00000032ab2f0f5d in clone () from /lib64/libc.so.6

        Show
        Ken Giusti added a comment - Here's a backtrace of the lockup: Thread 3 (Thread 0x7f49b3bff700 (LWP 24447)): #0 0x00000032ab60e34d in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00000032ab609fb2 in _L_lock_910 () from /lib64/libpthread.so.0 #2 0x00000032ab609e4b in pthread_mutex_lock () from /lib64/libpthread.so.0 [attempts to take Link->lock] #3 0x00007f49b4b91b19 in qpid::broker::Link::maintenanceVisit() () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidbroker.so.2 #4 0x00007f49b4b94db1 in qpid::broker::LinkTimerTask::fire() () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidbroker.so.2 [take timer->callbackLock] #5 0x00007f49b46ecfe1 in qpid::sys::Timer::fire(boost::intrusive_ptr<qpid::sys::TimerTask>) () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidcommon.so.2 #6 0x00007f49b46ee486 in qpid::sys::Timer::run() () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidcommon.so.2 #7 0x00007f49b4611d4a in qpid::sys::(anonymous namespace)::runRunnable(void*) () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidcommon.so.2 #8 0x00000032ab607d90 in start_thread () from /lib64/libpthread.so.0 #9 0x00000032ab2f0f5d in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x7f49b337d700 (LWP 24448)): #0 0x00000032ab60e34d in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00000032ab609fb2 in _L_lock_910 () from /lib64/libpthread.so.0 #2 0x00000032ab609e4b in pthread_mutex_lock () from /lib64/libpthread.so.0 [attempts to take timer->callbackLock] #3 0x00007f49b46ed202 in qpid::sys::TimerTask::cancel() () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidcommon.so.2 [takes link->lock] #4 0x00007f49b4b8f42e in qpid::broker::Link::destroy() () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidbroker.so.2 #5 0x00007f49b4b4ebd8 in qpid::broker::Connection::doIoCallbacks() () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidbroker.so.2 #6 0x00007f49b4b51526 in qpid::broker::Connection::doOutput() () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidbroker.so.2 #7 0x00007f49b4b21a0d in qpid::amqp_0_10::Connection::canEncode() () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidbroker.so.2 #8 0x00007f49b46e3b75 in qpid::sys::AsynchIOHandler::idle(qpid::sys::AsynchIO&) () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidcommon.so.2 #9 0x00007f49b460f47d in qpid::sys::posix::AsynchIO::writeable(qpid::sys::DispatchHandle&) () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidcommon.so.2 #10 0x00007f49b46e8796 in qpid::sys::DispatchHandle::processEvent(qpid::sys::Poller::EventType) () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidcommon.so.2 #11 0x00007f49b461730f in qpid::sys::Poller::run() () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidcommon.so.2 #12 0x00007f49b4611d4a in qpid::sys::(anonymous namespace)::runRunnable(void*) () from /home/kgiusti/Desktop/work/qpid/github/build/dyna-route-q/src/.libs/libqpidcommon.so.2 #13 0x00000032ab607d90 in start_thread () from /lib64/libpthread.so.0 #14 0x00000032ab2f0f5d in clone () from /lib64/libc.so.6
        Ken Giusti made changes -
        Field Original Value New Value
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Justin Ross made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Ken Giusti
            Reporter:
            Ken Giusti
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development