[YARN-10033] TestProportionalCapacityPreemptionPolicy not initializing vcores for effective max resources

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.0, 3.2.1, 3.1.3
• Fix Version/s: 3.3.0, 3.2.2, 3.1.4
• Component/s: capacity scheduler, test
• Assignee: Eric Payne
• Reporter: Eric Payne
• Created: 13/Dec/19 19:08
• Updated: 18/Dec/19 14:18
• Resolved: 17/Dec/19 17:50

Description

TestProportionalCapacityPreemptionPolicy#testPreemptionWithVCoreResource is preempting more containers than would happen on a real cluster.
This is because the process for mocking CS queues in TestProportionalCapacityPreemptionPolicy fails to take into consideration vcores when mocking effective max resources.
This causes miscalculations for how many vcores to preempt when the DRF is being used in the test:

TempQueuePerPartition#offer

    Resource absMaxCapIdealAssignedDelta = Resources.componentwiseMax(
        Resources.subtract(getMax(), idealAssigned),
        Resource.newInstance(0, 0));

In the above code, the preemption policy is offering resources to an underserved queue. getMax() will use the effective max resource if it exists. Since this test is mocking effective max resources, it will return that value. However, since the mock doesn't include vcores, the test treats memory as the dominant resource and awards too many preempted containers to the underserved queue.