Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25502

[Spark Job History] Empty Page when page number exceeds the reatinedTask size

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.3.1
    • Fix Version/s: 2.3.3, 2.4.0
    • Component/s: Web UI
    • Labels:
      None

      Description

      Steps:
      1. Spark installed and running properly.
      2. spark.ui.retainedTask=100000 ( it is default value )
      3.Launch Spark shell ./spark-shell --master yarn
      4. Create a spark-shell application with a single job and 500000 task
      val rdd = sc.parallelize(1 to 500000, 500000)
      rdd.count
      5. Launch Job History Page and go to spark-shell application created above under Incomplete Task
      6. Right click and got to Job page of the application and from there click and launch Stage Page
      7. Launch the Stage Id page for the specific Stage Id for the above created job
      8. Scroll down and check for the task completion Summary
      It Displays pagination panel showing 5000 Pages Jump to 1 Show 100 items in a page and Go button
      9. Replace 1 with 2333 page number
      Actual Result:
      2 Pagination Panel displayed
      Expected Result:
      Pagination Panel should not display 5000 pages as retainedTask value is 100000 and it should display 1000 page only because each page holding 100 tasks

        Attachments

          Activity

            People

            • Assignee:
              shahid shahid
              Reporter:
              abhishek.akg ABHISHEK KUMAR GUPTA
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: