[YARN-6144] FairScheduler: preempted resources can become negative

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha2
• Fix Version/s: 2.9.0, 3.0.0-alpha4
• Component/s: fairscheduler, resourcemanager
• Target Version/s: 2.9.0, 3.0.0-alpha4
• Hadoop Flags: Reviewed
• Assignee: Miklos Szegedi
• Reporter: Miklos Szegedi
• Created: 03/Feb/17 21:40
• Updated: 10/Feb/17 00:11
• Resolved: 09/Feb/17 23:15

Description

preemptContainers() calls trackContainerForPreemption() to collect the list of containers and resources that were preempted for an application. Later the list is reduced when containerCompleted() calls untrackContainerForPreemption(). The bug is that the resource variable preemptedResources is subtracted, not just when the container was preempted but also when it has completed successfully. This causes that we return an incorrect value in getResourceUsage().

Issue Links

• is broken by: YARN-4752 FairScheduler should preempt for a ResourceRequest and all preempted containers should be on the same node