[QPID-4672] C++ Broker deadlock detaching XmlExchange sessions

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.20
• Fix Version/s: 0.22
• Component/s: C++ Broker

Environment

C++ Broker, autotools build, make check or run_federation_tests

• Assignee: Unassigned
• Reporter: Charles E. Rolke
• Created: 27/Mar/13 13:14
• Updated: 29/Jul/13 18:54
• Resolved: 29/Apr/13 20:29

Description

Self test federation.FederationTests.test_dynamic_topic locks up. Pstack shows three threads with the same trace:

#0 pthread_rwlock_wrlock ()
#1 qpid::broker::XmlExchange::unbind()
#2 qpid::broker::XmlExchange::fedUnbind()
#3 qpid::broker::XmlExchange::bind()
#4 qpid::broker::Queue::bind()
#5 qpid::broker::Broker::bind()
#6 qpid::broker::SemanticState::unbindSessionBindings()
#7 qpid::broker::SemanticState::closed()
#8 qpid::broker::SessionState::~SessionState()
#9 qpid::broker::SessionState::~SessionState()
#10 qpid::broker::SessionHandler::handleDetach()
#11 qpid::amqp_0_10::SessionHandler::detach()

The lock occurs because there are two simultaneous session detach calls going on. Each takes out the Rlock and then tries to take out the Wlock. Neither will get the Wlock until everyone else releases their Rlock. Subsequent detach calls fall into the same catch as they get the Rlock but not the Wlock.

This condition is present in both the XmlExchange and DirectExchange. I'll have a patch for review shortly.