[AMQ-3176] Potential deadlock in duplex network connector recreation, resulting in dangling connections

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 5.4.2
• Fix Version/s: 5.5.0
• Component/s: Connector
• Labels: duplex networkconnector shutdown
• Assignee: Gary Tully
• Reporter: Gary Tully
• Created: 09/Feb/11 15:18
• Updated: 10/Feb/11 11:11
• Resolved: 10/Feb/11 11:11

Description

End result is lots of connections that are taking too long to shutdown and in particular:

2011-02-07 08:56:49,422 | INFO  | sport: tcp:///xxx:56982 | TransportConnection | The connection to '/xxx:56981' is taking a long time to shutdown.
....
2011-02-07 08:56:49,458 | INFO  | sport: tcp:///xxx:56981 | TransportConnection | The connection to '/xxx:56982' is taking a long time to shutdown.

where there is an overlap, with two connections trying to stop each other.
Problem appears when the initiator of a duplex network connector sees a failure and trys to reconnect and the responder sees the old transport connector in place. It tries to stop the existing connection but does it in a sync call so the potential to block and lock is present.

2011-01-26 16:35:54,618 [..] INFO TransportConnection - The connection to '/xx:51585' is taking a long time to shutdown.
2011-01-26 16:35:56,500 [..] INFO TransportConnection - The connection to '/xx:51585' is taking a long time to shutdown.

In particular,

Issue Links

• incorporates: AMQ-3129 Can only have one duplex networkConnection per transportConnection
• is broken by: AMQ-2774 Network of brokers : Multicast discovery stopped to work