[SPARK-25392] [Spark Job History]Inconsistent behaviour for pool details in spark web UI and history server page

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 2.4.5, 3.0.0
• Component/s: Spark Core

Environment

OS: SUSE 11

Spark Version: 2.3

• Assignee: shahid
• Reporter: ABHISHEK KUMAR GUPTA
• Created: 10/Sep/18 06:31
• Updated: 23/Jan/20 02:23
• Resolved: 16/Dec/19 23:07

Description

Steps:
1.Enable spark.scheduler.mode = FAIR
2.Submitted beeline jobs
create database JH;
use JH;
create table one12( id int );
insert into one12 values(12);
insert into one12 values(13);
Select * from one12;
3.Click on JDBC Incompleted Application ID in Job History Page
4. Go to Job Tab in staged Web UI page
5. Click on run at AccessController.java:0 under Desription column
6 . Click default under Pool Name column of Completed Stages table
URL:http://blr1000023109:23020/history/application_1536399199015_0006/stages/pool/?poolname=default
7. It throws below error
HTTP ERROR 400

Problem accessing /history/application_1536399199015_0006/stages/pool/. Reason:

Unknown pool: default

Powered by Jetty:// x.y.z

But under

Yarn resource page it display the summary under Fair Scheduler Pool: default
URL:https://blr1000023110:64323/proxy/application_1536399199015_0006/stages/pool?poolname=default

Summary
Pool Name Minimum Share Pool Weight Active Stages Running Tasks SchedulingMode
default 0 1 0 0 FIFO

Issue Links

• is duplicated by: SPARK-30483 Job History does not show pool properties table
• links to: [Github] Pull Request #22571 (sandeep-katta)
• links to: [Github] Pull Request #22571 (sandeep-katta)
• links to: GitHub Pull Request #22571
• links to: GitHub Pull Request #26616