[SPARK-2298] Show stage attempt in UI

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.1.0
• Component/s: Web UI
• Target Version/s: 1.1.0
• Assignee: Reynold Xin
• Reporter: Reynold Xin
• Created: 26/Jun/14 21:02
• Updated: 07/Jan/15 20:07
• Resolved: 27/Aug/14 07:34

Description

We should add a column to the web ui to show stage attempt id. Then tasks should be grouped by (stageId, stageAttempt) tuple.

When a stage is resubmitted (e.g. due to fetch failures), we should get a different entry in the web ui and tasks for the resubmission go there.

See the attached screenshot for the confusing status quo. We currently show the same stage entry twice, and then tasks appear in both.

Issue Links

• is related to: SPARK-2501 Handle stage re-submissions properly in the UI
• is related to: SPARK-2015 Spark UI issues at scale
• relates to: SPARK-2299 Consolidate various stageIdTo* hash maps
• links to: [Github] Pull Request #1384 (tsudukim)
• links to: [Github] Pull Request #1545 (rxin)