[YARN-4719] Add a helper library to maintain node state and allows common queries

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.8.0
• Fix Version/s: 2.9.0, 3.0.0-alpha1
• Component/s: scheduler
• Target Version/s: 2.9.0
• Hadoop Flags: Reviewed
• Assignee: Karthik Kambatla
• Reporter: Karthik Kambatla
• Created: 22/Feb/16 16:46
• Updated: 07/Nov/18 11:20
• Resolved: 14/Mar/16 21:32

Description

The scheduler could use a helper library to maintain node state and allowing matching/sorting queries. Several reasons for this:

1. Today, a lot of the node state management is done separately in each scheduler. Having a single library will take us that much closer to reducing duplication among schedulers.
2. Adding a filtering/matching API would simplify node labels and locality significantly.
3. An API that returns a sorted list for a custom comparator would help YARN-1011 where we want to sort by allocation and utilization for continuous/asynchronous and opportunistic scheduling respectively.

Issue Links

• causes: YARN-8969 AbstractYarnScheduler#getNodeTracker should return generic type to avoid type casting
• contains: YARN-6867 AbstractYarnScheduler reports the configured maximum resources, instead of the actual, even after the configured waittime