[YUNIKORN-2323] Gang scheduling user experience issues

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.4.0
• Fix Version/s: 1.6.0
• Component/s: core - scheduler
• Labels: pull-request-available
• Target Version: 1.6.0
• Assignee: Manikandan R
• Reporter: Manikandan R
• Created: 11/Jan/24 05:46
• Updated: 09/Sep/24 11:54
• Resolved: 09/Sep/24 11:54

Description

In case of any issues, users are finding it bit difficult to understand what is going on with the gang app. 

Issue 1:

"driver pod is getting struck"

At times, when driver pod is not able to run successfully for some reasons, users are getting the perspective that pod is getting struck and app is hanged, not moving further. Users are waiting for some time and don't understand the clear picture. How do we close the gap quickly and communicate accordingly through events?

Issue 2:

ResumeApplication is fired when all ph's are timed out. Do we need to inform the users about this event as they may not clue any about this significant change?

Issue 3: 

When Gang app ph's are in progress (and allocated), when there is request for real asks and there is resource crunch, do we need to trigger auto scaling?

Issue Links

• is related to: YUNIKORN-2653 Gang scheduling K8s event formatting compliance
• relates to: YUNIKORN-570 Display app's pending ask in the app's page
• links to: GitHub Pull Request #835
• links to: GitHub Pull Request #876