[AMQ-4561] Deadlock in DemandForwardingBridgeSupport class if ConsumerInfo message has arrived before BrokerInfo message.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 5.6.0, 5.7.0
• Component/s: Broker
• Labels: broker deadlock
• Patch Info: Patch Available
• Assignee: Unassigned
• Reporter: Yuriy Sidelnikov
• Created: 29/May/13 09:29
• Updated: 15/Jul/13 12:30
• Resolved: 08/Jul/13 20:01

Description

To establish a connection a spoke broker sends some special messages to the hub:
• BrokerInfo
• ConnectionInfo
• SessionInfo
• ProducerInfo
• ConsumerInfo
In return hub gsends exactly the same set of messages:
• BrokerInfo
• ConnectionInfo
• SessionInfo
• ProducerInfo
• ConsumerInfo

All messages are sent asynchronously so message order is not guaranteed.

However BrokerInfo message has to be delivered BEFORE ConsumerInfo message to the spoke broker because:
1. ConsumerInfo processing logic is depended on some information provided in BrokerInfo message so spoke broker side has lock on it.
2. All incoming messages are processed in the same thread including reading from tcp/nio stream.
Short excerpt from DemandForwardingBridgeSupport class:
private void startLocalBridge() throws Throwable

{  BrokerInfo message processing ….. localStartedLatch.countDown(); }

And ConsumerInfo message processing part has the lock:
case ConsumerInfo.DATA_STRUCTURE_TYPE:
localStartedLatch.await();
So if ConsumerInfo message has been received before BrokerInfo message the consuming thread will be in dead lock:
Name: ActiveMQ NIO Worker 13
State: WAITING on java.util.concurrent.CountDownLatch$Sync@150b06b
Total blocked: 1 Total waited: 3
Stack trace:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
java.util.concurrent.CountDownLatch.await(CountDownLatch.java:207)
org.apache.activemq.network.DemandForwardingBridgeSupport.serviceRemoteCommand(DemandForwardingBridgeSupport.java:534)

Issue Links

• is related to: AMQ-4632 Fix of AMQ-4328 causes deadlocks during set up of the bridge