[YUNIKORN-1559] rethink QOS BestEffort pod scheduling

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Implemented
• Fix Version/s: 1.3.0
• Component/s: core - scheduler, shim - kubernetes
• Labels: pull-request-available
• Target Version: 1.3.0
• Assignee: Qi Zhu
• Reporter: Wilfred Spiegelenburg
• Created: 06/Feb/23 01:34
• Updated: 20/Jun/23 14:41
• Resolved: 14/Mar/23 02:25

Description

The best effort pods on K8s do not request resources. The default scheduler will just dump them on a node and hope for the best.

In YuniKorn we need resources to schedule a pod. We reject a pod without a request set. As a default we assign 1M memory to the pod to make sure it gets scheduled. 1 byte would be better, need to rethink how we do this.

This also affects other BestEffort pod resource calculations

• init containers
• pod overhead 

Issue Links

• Is contained by: YUNIKORN-1632 Yunikorn fails to account for the max number of pods on a node
• is related to: YUNIKORN-1658 autoscaling is not triggered for requests with unlimited resources
• split from: YUNIKORN-1526 support K8s pod overhead
• links to: GitHub Pull Request #521
• links to: GitHub Pull Request #558