[SPARK-24819] Fail fast when no enough slots to launch the barrier stage on job submitted

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 2.4.0
• Component/s: Spark Core
• Epic Link: Support Barrier Execution Mode
• Assignee: Xingbo Jiang
• Reporter: Xingbo Jiang
• Created: 16/Jul/18 08:19
• Updated: 20/Sep/18 17:08
• Resolved: 15/Aug/18 20:31

Description

Check all the barrier stages on job submitted, to see whether the barrier stages require more slots (to be able to launch all the barrier tasks in the same stage together) than currently active slots in the cluster. If the job requires more slots than available (both busy and free slots), fail the job on submit.

Issue Links

• links to: [Github] Pull Request #22001 (jiangxb1987)