[YARN-6538] Inter Queue preemption is not happening when DRF is configured

• Type: Sub-task
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.8.0
• Component/s: capacity scheduler, scheduler preemption
• Labels: pull-request-available
• Assignee: Benjamin Teke
• Reporter: Sunil G
• Created: 28/Apr/17 18:15
• Updated: 30/Nov/22 12:21

Description

Cluster capacity of <memory:3TB, vCores:168>. Here memory is more and vcores are less. If applications have more demand, vcores might be exhausted.
Inter queue preemption ideally has to be kicked in once vcores is over utilized. However preemption is not happening.

Analysis:
In AbstractPreemptableResourceCalculator.computeFixpointAllocation,

    // assign all cluster resources until no more demand, or no resources are
    // left
    while (!orderedByNeed.isEmpty() && Resources.greaterThan(rc, totGuarant,
        unassigned, Resources.none())) {

will loop even when vcores are 0 (because memory is still +ve). Hence we are having more vcores in idealAssigned which cause no-preemption cases.

Issue Links

• links to: GitHub Pull Request #5174