[YUNIKORN-1740] [Umbrella] MockScheduler improvements

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.0
• Component/s: shim - kubernetes, test - unit
• Assignee: Peter Bacsko
• Reporter: Peter Bacsko
• Created: 16/May/23 15:33
• Updated: 20/Nov/23 16:42
• Resolved: 31/Jul/23 22:33

Description

With the MockScheduler object, it's possible to test Yunikorn as a whole without having an actual Kubernetes cluster.

Currently, MockScheduler has some limitations. For example:

• adding node only updates the core, not the shim
• recovery is skipped (ignored because of APIProvider.IsTestingMode())
• predicates are not called (ignored because of APIProvider.IsTestingMode())
• cannot simulate shared informer calls like AddPod(), UpdatePod(), etc.

However, it's possible to update the logic in such a way that we can simulate the entire lifecycle of an application (ie. all relevant code paths):

• New *v1.Pod object -> shim -> SI object -> core -> allocation
• allocation -> SI object -> shim -> bind pod
• Running state: updated *v1.Pod -> shim -> process pod object in the cache
• Completed/Failed state: updated *v1.Pod -> shim > process pod object in the cache -> SI object -> core -> update application state
• *v1.Node object -> shim -> update cache -> SI object -> core -> node&cluster update

This can also gives us a possibility to check the performance of Yunikorn and collect CPU&heap samples. This will not be a replacement of a real performance test, but with a proper test case, we can quickly do a rough assessment of performance/memory usage related changes.

Sub-Tasks

• Export methods in Context and always create PodEventHandler: Closed
• Add new methods to MockScheduler to handle Kubernetes objects: Closed
• Enable predicates for MockScheduler: Closed
• Enable recovery for MockScheduler: Closed
• Add implementation to MockedAPIProvider.AddEventHandler(): Closed
• Create test case which measures binding throughput using MockScheduler: Closed