Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25451

Stages page doesn't show the right number of the total tasks

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.1
    • Fix Version/s: 2.4.1, 3.0.0
    • Component/s: Web UI
    • Labels:
      None

      Description

       

      See the attached pic.

       

      The executor 1 has 7 tasks, but in the Stages Page the total tasks of executor is 6.

       

      to reproduce this simply start a shell:

      $SPARK_HOME/bin/spark-shell --executor-cores 1 --executor-memory 1g --total-executor-cores 2 --master spark://localhost.localdomain:7077

      Run job as fellows:

      sc.parallelize(1 to 10000, 3).map{ x => throw new RuntimeException("Bad executor")}.collect() 

       

      Go to the stages page and you will see the Total Tasks  is not right in

      Aggregated Metrics by Executor

      table. 

       

        Attachments

        1. mshot.png
          113 kB
          zuotingbing

          Activity

            People

            • Assignee:
              shahid shahid
              Reporter:
              zuo.tingbing9 zuotingbing
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: