[YARN-5774] MR Job stuck in ACCEPTED status without any progress in Fair Scheduler if set yarn.scheduler.minimum-allocation-mb to 0.

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha1
• Fix Version/s: 2.9.0, 3.0.0-alpha2
• Component/s: resourcemanager
• Labels: oct16-easy
• Target Version/s: 2.8.0, 3.0.0-alpha2
• Hadoop Flags: Reviewed
• Assignee: Yufei Gu
• Reporter: Yufei Gu
• Created: 24/Oct/16 21:09
• Updated: 22/Dec/16 22:33
• Resolved: 22/Dec/16 18:42

Description

MR Job stuck in ACCEPTED status without any progress in Fair Scheduler because there is no resource request for the AM. This happened when you configure yarn.scheduler.minimum-allocation-mb to zero.

The problem is in the code used by both Capacity Scheduler and Fair Scheduler. scheduler.increment-allocation-mb is a concept in FS, but not CS. So the common code in class RMAppManager passes the yarn.scheduler.minimum-allocation-mb as incremental one because there is no incremental one for CS when it tried to normalize the resource requests.

     SchedulerUtils.normalizeRequest(amReq, scheduler.getResourceCalculator(),
          scheduler.getClusterResource(),
          scheduler.getMinimumResourceCapability(),
          scheduler.getMaximumResourceCapability(),
          scheduler.getMinimumResourceCapability());  --> incrementResource should be passed here.