[YUNIKORN-1385] To provide visibility to application's aggregated resource consumption

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.3.0
• Component/s: core - common, core - scheduler, shim - kubernetes
• Labels: pull-request-available
• Target Version: 1.3.0
• Assignee: Yongjun Zhang
• Reporter: Yongjun Zhang
• Created: 04/Nov/22 21:06
• Updated: 24/Nov/23 04:26
• Resolved: 24/May/23 19:24

Description

Currently the "Used Resource" for a given application reported at Yunikorn GUI is actually the snapshot of the resources currently allocated to the application. We need to provide visibility to how much resources an application has used so far, and it would be important to report  how much total resources is used by the application. 

The unit of resources used can be memoryseconds/vcoreseconds like how Hadoop Yarn does in its app summary report. However, there is more complexity with K8s which supports different instance types (Hadoop Yarn does support multiple instance types but its app summary report ignored instance types). It would be nice to report how much resource is used for each instance types used by an application.

This is a top level Jira for this new feature. Subtask jira will be created when we work on it.

Issue Links

• Blocked: YUNIKORN-2151 Report resource used by placeholder pods in the app summary
• causes: YUNIKORN-1889 Remove instanceType from scheduling cycle
• relates to: YUNIKORN-1389 Pass node labels from shim to core.
• relates to: YUNIKORN-1549 Introduce a config to specify what attribute to use as instance type
• relates to: YUNIKORN-1565 [Umbrella] Node attribute related improvement and bug fix.
• links to: Design Doc 0.1
• links to: GitHub Pull Request #507