[YUNIKORN-201] [Umbrella] Application tracking API and CRD - phase 1

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.10
• Component/s: core - scheduler, scheduler-interface, shim - kubernetes
• Target Version: 0.10
• Assignee: Kinga Marton
• Reporter: Weiwei Yang
• Created: 03/Jun/20 06:15
• Updated: 05/Sep/23 16:35
• Resolved: 24/Mar/21 15:35

Description

Today, YK works behind the scene, and the workflow is like

1. app operator or job server launch a bunch of pods on K8s
2. YK gets notified and group pods to apps based on appID
3. YK schedules the pods with respect to the app info

This provides a simple model to integrate with existing K8s and to support workloads, but it has some user experience issues. Such as

1. YK can hardly manage the app lifecycle end to end. An outstanding issue is we do not know when an app is finished if we only look at the pod status. 
2. YK doesn't have ability to admit apps. We need the ability to admit app based on various conditions, e.g resource quota, cluster overhead, ACL, etc. 
3. Hard to track app status. Sometimes app might be pending in resource queues, but we do not have a good way to expose such status info.

To further improve the user experience, we need to introduce an application tracking API and K8s custom resource definition (CRD). The CRD will be used by app operator/job server to interact with YK, to get the lifecycle fully controlled.

Issue Links

• is blocked by: YUNIKORN-373 [umbrella] Upgrade Kubernetes dependency to 1.16
• links to: Design doc

Sub-Tasks

• Create YuniKorn App CRD: Closed
• Implement basic controller logic: Closed
• Include CRD registration into Helm charts: Closed
• When deleting a deployment the application is not deleted from the shim cache: Closed
• Create App CRD definition and example: Closed
• Add e2e test to verify app CRD: Closed
• App CRD definition should support root.default format: Closed
• Add metadta.UID selector for the App CRD: Closed
• Register the CRD programatically if it is turned on: Closed
• Remove installCRD option during helm install for e2e test: Closed
• Update latest helm chart version on helm hub: Closed
• Pass applicationID for spark pods: Closed
• Make sure when there is no allocation in an app, the app state is "Waiting".: Closed