[SPARK-4145] Create jobs overview and job details pages on the web UI

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.2.0
• Component/s: Web UI
• Target Version/s: 1.2.0
• Assignee: Josh Rosen
• Reporter: Josh Rosen
• Created: 30/Oct/14 01:02
• Updated: 13/Dec/14 22:38
• Resolved: 24/Nov/14 21:19

Description

We should create a jobs overview and job details page on the web UI. The overview page would list all jobs in the SparkContext and would replace the current "Stages" page as the default web UI page. The job details page would provide information on the stages triggered by a particular job; it would also serve as a place to show DAG visualizations and other debugging aids.

I still plan to keep the current "Stages" page, which lists all stages of all jobs, since it's a useful debugging aid for figuring out how resources are being consumed across all jobs in a Spark Cluster.

Issue Links

• blocks: SPARK-3468 Provide timeline view in Job UI pages
• blocks: SPARK-3682 Add helpful warnings to the UI
• blocks: SPARK-4411 Add "kill" link for jobs in the UI
• incorporates: SPARK-2642 Add jobId in web UI
• is duplicated by: SPARK-1362 Web UI should provide page of showing statistics and stage list for a given job
• links to: [Github] Pull Request #3009 (JoshRosen)