[SPARK-26341] Expose executor memory metrics at the stage level, in the Stages tab

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.1
• Fix Version/s: 3.1.0, 3.2.0
• Component/s: Spark Core, Web UI
• Assignee: angerszhu
• Reporter: Edward Lu
• Created: 11/Dec/18 20:49
• Updated: 19/Dec/20 20:53
• Resolved: 18/Dec/20 05:34

Description

Sub-task SPARK-23431 will add stage level executor memory metrics (peak values for each stage, and peak values for each executor for the stage). This information should also be exposed the the web UI, so that users can see which stages are memory intensive.

Issue Links

• links to: [Github] Pull Request #30573 (AngersZhuuuu)
• links to: [Github] Pull Request #30858 (sarutak)
• links to: [Github] Pull Request #30858 (sarutak)
• links to: GitHub Pull Request #28036