[MAPREDUCE-5822] FairScheduler does not preempt due to fairshare-starvation when fairshare is 1

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2.1
• Fix Version/s: 1.3.0
• Component/s: scheduler
• Hadoop Flags: Reviewed
• Assignee: Anubhav Dhoot
• Reporter: Anubhav Dhoot
• Created: 07/Apr/14 22:42
• Updated: 11/Apr/14 21:53
• Resolved: 11/Apr/14 21:53

Description

If the fair share returned by the scheduler getFairShare() == 1 the pool will never be marked as being starved because of the following calculation:

boolean isStarvedForFairShare(PoolSchedulable sched) { 
  int desiredFairShare = (int) Math.floor(Math.min( sched.getFairShare() / 2, sched.getDemand())); 
  return (sched.getRunningTasks() < desiredFairShare); 
}

getFairShare() returns 1
Math.min calculation will return 0.5
Math.Floor() which will cause the desiredFairShare to be set to 0.
the return value to be 'false' (0 < 0)
If you have a small job without a minimum set it will not get scheduled if a large job is hogging the slots.