[YARN-4496] Improve HA ResourceManager Failover detection on the client

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.9.0, 3.0.0-alpha1
• Component/s: client, resourcemanager
• Assignee: Jian He
• Reporter: Arun Suresh
• Created: 23/Dec/15 02:21
• Updated: 05/Oct/16 22:17
• Resolved: 23/Jan/16 02:22

Description

HDFS deployments can currently use the RequestHedgingProxyProvider to improve Namenode failover detection in the client. It does this by concurrently trying all namenodes and picks the namenode that returns the fastest with a successful response as the active node.

It would be useful to have a similar ProxyProvider for the Yarn RM (it can possibly be done by converging some the class hierarchies to use the same ProxyProvider)

This would especially be useful for large YARN deployments with multiple standby RMs where clients will be able to pick the active RM without having to traverse a list of configured RMs.

Issue Links

• relates to: YARN-5711 Propogate exceptions back to client when using hedging RM failover provider