[SPARK-15926] Improve readability of DAGScheduler stage creation methods

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.1.0
• Component/s: Scheduler, Spark Core
• Target Version/s: 2.1.0
• Assignee: Kay Ousterhout
• Reporter: Kay Ousterhout
• Created: 13/Jun/16 19:35
• Updated: 17/May/20 17:47
• Resolved: 17/Jun/16 19:14

Description

The current code to create new stages is very confusing: it's difficult to reason about which functions actually create new stages versus just looking up existing stages, and there are many similarly-named functions that do very different things. The goal of this JIRA is to clean some of that up.

Issue Links

• is duplicated by: SPARK-15422 Remove unnecessary calculation of stage's parents
• links to: [Github] Pull Request #13677 (kayousterhout)