[YARN-736] Add a multi-resource fair sharing metric

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.4-alpha
• Fix Version/s: 2.1.0-beta
• Component/s: scheduler
• Hadoop Flags: Reviewed
• Assignee: Sandy Ryza
• Reporter: Sandy Ryza
• Created: 30/May/13 20:44
• Updated: 27/Aug/13 22:15
• Resolved: 24/Jun/13 18:36

Description

Currently, at a regular interval, the fair scheduler computes a fair memory share for each queue and application inside it. This fair share is not used for scheduling decisions, but is displayed in the web UI, exposed as a metric, and used for preemption decisions.

With DRF and multi-resource scheduling, assigning a memory share as the fair share metric to every queue no longer makes sense. It's not obvious what the replacement should be, but probably something like fractional fairness within a queue, or distance from an ideal cluster state.

Issue Links

• depends upon: YARN-326 Add multi-resource scheduling to the fair scheduler