[YUNIKORN-1218] Scheduler crashed with concurrent map access error in health checker

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.1.0
• Component/s: core - scheduler
• Labels: pull-request-available
• Target Version: 1.1.0
• Assignee: Weiwei Yang
• Reporter: Weiwei Yang
• Created: 18/May/22 21:02
• Updated: 13/Sep/22 14:31
• Resolved: 18/May/22 21:25

Description

After YUNIKORN-1107, the health checker runs as a background thread in 30s interval. We observed a few scheduler restarts in the past week that seems to be caused by this thread, because it has an unsafe access to the partition context without proper read lock. I have uploaded a patch to reproduce this locally, and a file of the stack trace when crash happens.

Issue Links

• links to: GitHub Pull Request #411