Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
None
-
None
-
None
Description
Connection establishment may deadlock. Both client and server negotiation tasks are blocking. Assume there is only one thread available for negotiation processing. If servers A and B try to connect to each other, and submit the client-side task to their threads first, the server-side task required to complete those tasks will be blocked behind the client tasks, and will never be executed - hence the client tasks will never complete.
This happens in practice on a 20-node cluster which tries to connect the complete graph of connections between nodes in a burst, with 4 threads available for negotiation.
An easy fix is to partition the resources for server and client negotiations, by using separate thread pools.