[QPID-3216] Deadlock between "_lock" (in AMQSession#DisptcherThread) and "_messageDeliveryLock" (in AMQSession.java)

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 0.10
• Component/s: JMS AMQP 0-x
• Labels: deadlock
• Assignee: Unassigned
• Reporter: Rajith Muditha Attapattu
• Created: 19/Apr/11 15:11
• Updated: 10/Feb/15 09:53
• Resolved: 10/Feb/15 09:53

Description

As per the following thread dump you can clearly see the deadlock between "_lock" (in AMQSession#DisptcherThread) and "_messageDeliveryLock" (in AMQSession.java)
This is another regression and I am working to identify the rev that caused it and possibly understand the circumstances surrounding the change.

dispatcher thread already obtained "_lock" in dispatchMessage method.
dispatcher thread waiting to take "_messageDeliveryLock" in the same method.

main thread has already obtained "_messageDeliveryLock" in AMQSession.close method.
main thread waiting to take "_lock" in DispatcherThread.rejectPending method.

  Found one Java-level deadlock:
  =============================
  "Dispatcher-Channel-0":
    waiting to lock monitor 0x0000002ac1e6f0b8 (object 0x0000002a9f040448, a java.lang.Object),
    which is held by "main"
  "main":
    waiting to lock monitor 0x0000002ac1e6f010 (object 0x0000002a9f03ba40, a java.lang.Object),
    which is held by "Dispatcher-Channel-0"

  Java stack information for the threads listed above:
  ===================================================
  "Dispatcher-Channel-0":
     at org.apache.qpid.client.AMQSession$Dispatcher.dispatchMessage(AMQSession.java:3315)
     - waiting to lock<0x0000002a9f040448>  (a java.lang.Object)
     - locked<0x0000002a9f03ba40>  (a java.lang.Object)
     at org.apache.qpid.client.AMQSession$Dispatcher.access$900(AMQSession.java:3096)
     at org.apache.qpid.client.AMQSession.dispatch(AMQSession.java:3089)
     at org.apache.qpid.client.message.UnprocessedMessage.dispatch(UnprocessedMessage.java:55)
     at org.apache.qpid.client.AMQSession$Dispatcher.run(AMQSession.java:3243)
     at java.lang.Thread.run(Thread.java:619)
  "main":
     at org.apache.qpid.client.AMQSession$Dispatcher.rejectPending(AMQSession.java:3122)
     - waiting to lock<0x0000002a9f03ba40>  (a java.lang.Object)
     at org.apache.qpid.client.AMQSession.confirmConsumerCancelled(AMQSession.java:897)
     at org.apache.qpid.client.BasicMessageConsumer_0_10.sendCancel(BasicMessageConsumer_0_10.java:189)
     at org.apache.qpid.client.BasicMessageConsumer.close(BasicMessageConsumer.java:573)
     - locked<0x0000002a9ef30aa0>  (a java.lang.Object)
     at org.apache.qpid.client.AMQSession.closeConsumers(AMQSession.java:2464)
     at org.apache.qpid.client.AMQSession.closeProducersAndConsumers(AMQSession.java:2514)
     at org.apache.qpid.client.AMQSession.close(AMQSession.java:731)
     - locked<0x0000002a9f040448>  (a java.lang.Object)
     - locked<0x0000002a9ef30aa0>  (a java.lang.Object)
     at org.apache.qpid.client.AMQSession.close(AMQSession.java:708)
     at org.apache.qpid.client.AMQConnection.closeAllSessions(AMQConnection.java:1003)
     at org.apache.qpid.client.AMQConnection.doClose(AMQConnection.java:897)
     - locked<0x0000002a9ef30aa0>  (a java.lang.Object)
     - locked<0x0000002a9ef30ab0>  (a java.lang.Object)
     at org.apache.qpid.client.AMQConnection.doClose(AMQConnection.java:886)
     - locked<0x0000002a9f040448>  (a java.lang.Object)
     - locked<0x0000002a9ef30ab0>  (a java.lang.Object)
     at org.apache.qpid.client.AMQConnection.close(AMQConnection.java:870)
     at org.apache.qpid.client.AMQConnection.close(AMQConnection.java:861)
     at org.apache.qpid.client.AMQConnection.close(AMQConnection.java:856)
     at org.apache.qpid.test.utils.QpidBrokerTestCase.tearDown(QpidBrokerTestCase.java:1118)
     at junit.framework.TestCase.runBare(TestCase.java:130)
     at org.apache.qpid.test.utils.QpidBrokerTestCase.runBare(QpidBrokerTestCase.java:234)
     at junit.framework.TestResult$1.protect(TestResult.java:106)
     at junit.framework.TestResult.runProtected(TestResult.java:124)
     at junit.framework.TestResult.run(TestResult.java:109)
     at junit.framework.TestCase.run(TestCase.java:118)
     at org.apache.qpid.test.utils.QpidTestCase.run(QpidTestCase.java:120)
     at junit.framework.TestSuite.runTest(TestSuite.java:208)
     at junit.framework.TestSuite.run(TestSuite.java:203)
     at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:297)
     at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:672)
     at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:546)

Issue Links

• relates to: QPID-3207 AMQSession.java close() does not close the consumers with the correct arguments