[YUNIKORN-549] Scheduler recovery failure occasionally while recovering a large number of applications

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.10
• Component/s: shim - kubernetes
• Labels: pull-request-available
• Target Version: 0.10
• Assignee: Weiwei Yang
• Reporter: Weiwei Yang
• Created: 25/Feb/21 01:15
• Updated: 21/Jan/22 21:48
• Resolved: 25/Feb/21 09:19

Description

Current recovery logic adds application back to based on the pods reported by the informers/listers. In some conditions, the recovery of an app could fail if the app has both Running and Pending pods. This is because the shim marks an app with a Recovery if the informer notified the scheduler before the lister function gets called. The is not working as expected consistently, we need a stable implementation in order to tell if an app needs recovery or not.

Issue Links

• links to: GitHub Pull Request #235