[SPARK-32135] Show Spark Driver name on Spark history web page

• Type: Wish
• Status: In Progress
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 2.4.5
• Component/s: Spark Core
• Assignee: Unassigned
• Reporter: Gaurangi Saxena
• Created: 30/Jun/20 02:25
• Updated: 08/Sep/20 16:31

Description

Our service dynamically creates short-lived YARN clusters in cloud. Spark applications run on these dynamically created clusters. Log data for these applications is stored on a remote file-system. We configure a static instance of SparkHistoryServer to view information on jobs that ran on these clusters.

Since we are using a single History Server for multiple clusters, it will be useful for users to have information on where the jobs were executed. We would like to display this information on the main web-page, instead of having users go through multiple links to retrieve this information.

Issue Links

• links to: [Github] Pull Request #29629 (Gaurangi94)
• links to: [Github] Pull Request #29629 (Gaurangi94)