Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25392

[Spark Job History]Inconsistent behaviour for pool details in spark web UI and history server page

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 2.4.5, 3.0.0
    • Component/s: Spark Core
    • Labels:
      None
    • Environment:

      OS: SUSE 11

      Spark Version: 2.3

      Description

      Steps:
      1.Enable spark.scheduler.mode = FAIR
      2.Submitted beeline jobs
      create database JH;
      use JH;
      create table one12( id int );
      insert into one12 values(12);
      insert into one12 values(13);
      Select * from one12;
      3.Click on JDBC Incompleted Application ID in Job History Page
      4. Go to Job Tab in staged Web UI page
      5. Click on run at AccessController.java:0 under Desription column
      6 . Click default under Pool Name column of Completed Stages table
      URL:http://blr1000023109:23020/history/application_1536399199015_0006/stages/pool/?poolname=default
      7. It throws below error
      HTTP ERROR 400

      Problem accessing /history/application_1536399199015_0006/stages/pool/. Reason:

      Unknown pool: default

      Powered by Jetty:// x.y.z

      But under

      Yarn resource page it display the summary under Fair Scheduler Pool: default
      URL:https://blr1000023110:64323/proxy/application_1536399199015_0006/stages/pool?poolname=default

      Summary
      Pool Name Minimum Share Pool Weight Active Stages Running Tasks SchedulingMode
      default 0 1 0 0 FIFO

        Attachments

          Activity

            People

            • Assignee:
              shahid shahid
              Reporter:
              abhishek.akg ABHISHEK KUMAR GUPTA
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: