[SPARK-4655] Split Stage into ShuffleMapStage and ResultStage subclasses

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.0
• Component/s: Spark Core
• Target Version/s: 1.4.0
• Assignee: Ilya Ganelin
• Reporter: Josh Rosen
• Created: 29/Nov/14 19:45
• Updated: 01/Apr/15 10:09
• Resolved: 01/Apr/15 10:09

Description

The scheduler's Stage class has many fields which are only applicable to result stages or shuffle map stages. As a result, I think that it makes sense to make Stage into an abstract base class with two subclasses, ResultStage and ShuffleMapStage. This would improve the understandability of the DAGScheduler code.

Issue Links

• links to: [Github] Pull Request #4703 (ilganeli)
• links to: [Github] Pull Request #4708 (ilganeli)