[YUNIKORN-2322] Investigate YuniKorn stuck when scheduling latency is high

• Type: Task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Component/s: core - common
• Assignee: Rainie Li
• Reporter: Rainie Li
• Created: 11/Jan/24 00:32
• Updated: 10/Mar/24 06:49
• Resolved: 10/Mar/24 06:49

Description

We are seeing service stuck when latency increases, even cluster has resource, YuniKorn will not be able to schedule apps. We have to manually restart YuniKorn.

we did profiling to find out most time are used by tryReservedAllocate. 

Attached ** profiling screenshot and service latency data.