[QPID-4574] [JMS] Deadlock involving _failoverMutex and _messageDeliveryLock

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.14, 0.16, 0.18, 0.20
• Fix Version/s: 0.31
• Component/s: JMS AMQP 0-x
• Labels: deadlock
• Assignee: Rajith Muditha Attapattu
• Reporter: Rajith Muditha Attapattu
• Created: 08/Feb/13 16:29
• Updated: 19/Mar/15 12:22
• Resolved: 10/Feb/15 09:43

Description

This deadlock can manifest when a you have a producer sending messages inside an onMessage() call, while another thread is either closing the session or the consumer.

Sending a message inside onMessage() is a common enough pattern, and is used by intermediaries like Message Bridges, ESB's etc..

Here is an example stack trace, showing deadlock occuring when closing the consumer:

"Dispatcher-0-Conn-1":
	at org.apache.qpid.client.BasicMessageProducer.send(BasicMessageProducer.java:309)
	- waiting to lock <0x00000000ecfd15a8> (a java.lang.Object)
	at Test_00782235$1.onMessage(Test_00782235.java:29)
	at org.apache.qpid.client.BasicMessageConsumer.notifyMessage(BasicMessageConsumer.java:751)
	at org.apache.qpid.client.BasicMessageConsumer_0_10.notifyMessage(BasicMessageConsumer_0_10.java:141)
	at org.apache.qpid.client.BasicMessageConsumer.notifyMessage(BasicMessageConsumer.java:725)
	at org.apache.qpid.client.BasicMessageConsumer_0_10.notifyMessage(BasicMessageConsumer_0_10.java:186)
	at org.apache.qpid.client.BasicMessageConsumer_0_10.notifyMessage(BasicMessageConsumer_0_10.java:54)
	at org.apache.qpid.client.AMQSession$Dispatcher.notifyConsumer(AMQSession.java:3479)
	at org.apache.qpid.client.AMQSession$Dispatcher.dispatchMessage(AMQSession.java:3418)
	- locked <0x00000000ecfd16c8> (a java.lang.Object)
	- locked <0x00000000ecfd16d8> (a java.lang.Object)
	at org.apache.qpid.client.AMQSession$Dispatcher.access$1000(AMQSession.java:3205)
	at org.apache.qpid.client.AMQSession.dispatch(AMQSession.java:3198)
	at org.apache.qpid.client.message.UnprocessedMessage.dispatch(UnprocessedMessage.java:54)
	at org.apache.qpid.client.AMQSession$Dispatcher.run(AMQSession.java:3341)
	at java.lang.Thread.run(Thread.java:679)

"main":
	at org.apache.qpid.client.BasicMessageConsumer.close(BasicMessageConsumer.java:598)
	- waiting to lock <0x00000000ecfd16c8> (a java.lang.Object)
	- locked <0x00000000ecfd15a8> (a java.lang.Object)
	at org.apache.qpid.client.BasicMessageConsumer.close(BasicMessageConsumer.java:558)
	at Test_00782235.<init>(Test_00782235.java:62)
	at Test_00782235.main(Test_00782235.java:12)

Issue Links

• is part of: QPID-6374 [Java Client] Eliminate deadlocks by acquiring locks in a consistent order
• is related to: QPID-5117 Java client deadlocks if connection is closed while onMessage() is creating a session
• is related to: QPID-5119 [JMS] 3-thread deadlock on pre-AMQP-0-10 connection involving messageDeliveryLock, failover mutex and sessionCreationLock
• relates to: QPID-5118 [JMS] Deadlock when closing a consumer and calling JMS operatons inside onMessage() on AMQP 0-8/0-9/0-9-1 connection