[HDFS-13633] RBF: Synchronous way to create RPC client connections to NN

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: federation
• Assignee: CR Hota
• Reporter: CR Hota
• Created: 29/May/18 18:15
• Updated: 25/Jun/19 02:37

Description

Currently the router code does the following.

1. IPC handler thread gets a connection from the pool, even if the connection is NOT usable.
2. At the same time the IPC thread also submits a request to connection creator thread for adding a new connection to the pool asynchronously.
3. The new connection is NOT utilized by the IPC threads that get back an unusable connection.

With this approach burst behaviors of clients, fill up the pool without necessarily using the connections. Also the approach is indeterministic.

We propose a flag that can allow router admins to control the behavior of getting connections by the IPC handler threads. The flag would allow to toggle ON/OFF asynchronous vs synchronous way of connection creation.

In the new model, if a connection is unusable, IPC handler thread would go ahead and create a connection and add to the pool and utilize it subsequently. It would still utilize the unusable connection if the pool is full.