[SPARK-40428] Add a shutdownhook to CoarseGrained scheduler to avoid dangling resources during abnormal shutdown

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: Kubernetes, Spark Core
• Assignee: Holden Karau
• Reporter: Holden Karau
• Created: 14/Sep/22 18:36
• Updated: 05/Oct/22 00:29
• Resolved: 05/Oct/22 00:29

Description

    Add a shutdown hook in the CoarseGrainedSchedulerBackend to call stop since we've got zombie pods hanging around since the resource tie isn't perfect.

Issue Links

• links to: [Github] Pull Request #37885 (holdenk)
• links to: [Github] Pull Request #37885 (holdenk)