[YUNIKORN-2] Support Gang Scheduling

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.10
• Component/s: core - scheduler
• Labels: pull-request-available
• Target Version: 0.10
• Assignee: Weiwei Yang
• Reporter: Weiwei Yang
• Created: 11/Feb/20 23:51
• Updated: 21/Jan/22 21:48
• Resolved: 24/Mar/21 15:36

Description

Gang scheduling is one of the most important features for schedulers, it is very useful for machine learning workloads such as tensorflow, pytorch, etc. Since yunikorn has notion of the application, it is not very hard for us to support gang scheduling context.

This umbrella tracks the efforts to support gang scheduling feature in yunikorn.

Issue Links

• Parent Feature: YUNIKORN-553 [Umbrella] Gang scheduling enhancements
• relates to: YUNIKORN-577 Correctly handle ask releases on TIMEOUT
• links to: Design Doc (WIP)
• links to: Detailed implementation design
• links to: GitHub Pull Request #215
• links to: GitHub Pull Request #219
• links to: Implementation design