[YUNIKORN-147] Extensive resource quota management

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.11
• Component/s: core - cache, core - common, core - scheduler, scheduler-interface, shim - kubernetes
• Assignee: Weiwei Yang
• Reporter: Weiwei Yang
• Created: 11/May/20 21:51
• Updated: 21/Jan/22 21:48
• Resolved: 07/Jan/22 01:05

Description

Today, in K8s. The resource quota is enforced by the quota admission controller. Resource quota is charged once a pod is created in a namespace, no matter it is running, pending (a pod only consumes resources when it is running), failed, or completed. The admission controller will reject further pods if all quota is exceeded.

When we run batch workloads with such quota, this can lead to some issues. Let's use Spark as an example: Spark job pods could be pending for many reasons, volume not ready, picky on a host, etc. Such pending pods will consume the resource quota. And subsequentially causing the resources can not be efficiently used.

The solution is to leverage YuniKorn elastic queues for quota management. Such elastic nature will be helpful to provide efficient resource sharing for batch workloads.

Sub-Tasks

• Define API in scheduler interface to queue administration: Closed
• Watch K8s namespace and create unmanaged queues accordingly: Closed