[YARN-8697] LocalityMulticastAMRMProxyPolicy should fallback to random sub-cluster when cannot resolve resource

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.2.0
• Assignee: Botong Huang
• Reporter: Botong Huang
• Created: 22/Aug/18 00:43
• Updated: 28/Aug/18 23:26
• Resolved: 28/Aug/18 23:03

Description

Right now in LocalityMulticastAMRMProxyPolicy, whenever we cannot resolve the resource name (node or rack), we always route the request to home sub-cluster. However, home sub-cluster might not be always be ready to use (timed out YARN-8581) or enabled (by AMRMProxyPolicy weights). It might also be overwhelmed by the requests if sub-cluster resolver has some issue. In this Jira, we are changing it to pick a random active and enabled sub-cluster for resource request we cannot resolve.