Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25503

[Spark Job History] Total task message in stage page is ambiguous

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.1
    • Fix Version/s: 2.3.3, 2.4.0
    • Component/s: Web UI
    • Labels:
      None

      Description

      Steps:
      1. Spark installed and running properly.
      2. spark.ui.retainedTask=100000 ( it is default value )
      3.Launch Spark shell ./spark-shell --master yarn
      4. Create a spark-shell application with a single job and 500000 task
      val rdd = sc.parallelize(1 to 500000, 500000)
      rdd.count
      5. Launch Job History Page and go to spark-shell application created above under Incomplete Task
      6. Right click and got to Job page of the application and from there click and launch Stage Page
      7. Launch the Stage Id page for the specific Stage Id for the above created job
      8. Scroll down and check for the task msg above Pagination Panel
      It Displays Task( 100000, Showing 500000)

      Actual Result:
      It displayed Task( 100000, Showing 500000)

      Expected Result:
      Since retainedTask=100000 and it should show 100000 task
      So message should be Task( 500000, Showing 100000)
       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                shahid shahid
                Reporter:
                abhishek.akg ABHISHEK KUMAR GUPTA
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: