[SPARK-27565] Show job info of WholeStageCodegen node on SparkSQL UI page

• Type: Improvement
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.1.0
• Component/s: SQL, Web UI
• Assignee: Unassigned
• Reporter: peng bo
• Created: 25/Apr/19 12:11
• Updated: 16/Mar/20 22:53

Description

Currently it's really hard to link SQL plan to Spark jobs on SparkSQL UI page. When one job failed, which PhysicalPlan nodes are involved are difficult to be located. This PR is to show the job info of WholeStageCodegen node on SparkSQL UI page to make life easier.
Each WholeStageCodegen node has a task metric PIPELINE_DURATION_METRIC which records the total duration time shown in the top right. The idea is to put the job info into its metric value during metric aggregation.

Issue Links

• links to: GitHub Pull Request #24460