[YUNIKORN-1628] [Umbrella] YuniKorn application traceability

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.0
• Component/s: core - scheduler
• Labels: release-notes
• Target Version: 1.4.0
• Assignee: Peter Bacsko
• Reporter: Peter Bacsko
• Created: 11/Mar/23 09:34
• Updated: 22/Feb/24 05:44
• Resolved: 14/Nov/23 15:56

Description

The current implementation of YuniKorn is focused on the application and the states of the application. K8s does not and cannot provide details on what happens inside the application. This limits what we can offer at a YuniKorn level for applications.

To increase supportability, we need to understand what happens inside the core scheduler and how we got into a certain state.

Requirements:

1. We want to record a stream of events in memory when something relevant happens which is related to the application or nodes:
   • Partition changed (nodes added / removed, capacity changed, etc.)
   • Application created / removed
   • An ask is created / removed
   • An allocation is created / removed
   • Reservation occurs
   • Placeholder is replaced, etc.
2. The recorded events should be available from the REST interface
3. The number of stored events can be limited by two settings: maximum number of events or expiration time (eg. events from the past 5 minutes).
4. Take advantage of Go channels to avoid any potential blocking

 

Issue Links

• blocks: YUNIKORN-2017 [Umbrella] YuniKorn application traceability - Phase 2
• is related to: YUNIKORN-2442 Documentation update about the event system
• relates to: YUNIKORN-2115 [Umbrella] YuniKorn application traceability - phase II
• relates to: YUNIKORN-1702 Application is removed from Yunikorn UI upon completion, better to keep for additional time