[YUNIKORN-2099] [Umbrella] State initialisation simplification (phase 2)

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Component/s: core - scheduler, shim - kubernetes
• Labels: release-notes
• Target Version: 1.5.0
• Assignee: Craig Condit
• Reporter: Craig Condit
• Created: 01/Nov/23 19:51
• Updated: 20/Mar/24 14:32
• Resolved: 04/Jan/24 18:13

Description

Startup rebuilds all state of the cluster. This is called recovery. The name is a bit misleading as it is not really recovery as it is loading the current state. State initialisation is a better term to use.

The current recovery code links the loading of applications and tasks (pods) to node loading. This makes the recovery code complex and thus fragile. It could, in a worst case scenario, lead to a pod not being recovered correctly.

Recovery should be a step by step process that has boundaries and steps:

• load node
  • register nodes with the core
• load pods
  • create applications in core
  • register running pods as allocations with the core
  • register pending pods as asks with the core
• process changes for nodes and pods
• start scheduling

No nodes, applications or asks on existing apps should be declined. Even if the  queue does not exist a running application must be added and handled. The current rejection of an application if it cannot be placed in the queue is an incorrect behaviour.

Issue Links

• split from: YUNIKORN-1777 [Umbrella] State initialisation simplification (phase 1)

Sub-Tasks

• Clean up AppManager implementation: Closed
• Clean up scheduler state initialization: Closed
• Add missing Originator and PreemptionPolicy fields to SI Allocation: Closed
• Populate Originator and PreemptionPolicy on existing allocations: Closed