[YUNIKORN-564] Improve the upscaling performance for StateAware policy

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.10
• Component/s: core - scheduler
• Labels: pull-request-available
• Target Version: 0.10
• Assignee: Weiwei Yang
• Reporter: Weiwei Yang
• Created: 12/Mar/21 07:04
• Updated: 21/Jan/22 21:48
• Resolved: 15/Mar/21 00:57

Description

When calculating the outstanding requests that potentially need to trigger the upscaling, we use the same sortingApp function that used by the core scheduling logic. When the sorting policy is StateAware, this policy filters out apps to limit num of apps that can be scheduled at each moment. This is designed for the scheduling phase, however, for trigging the upscaling, we do not need to filter those apps, as long as the requests can fit into the queues' headroom, we should allow them to trigger the upscaling. This will optimize the upscaling perf when the StateAware policy is used.

Issue Links

• links to: GitHub Pull Request #255