Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-20591

Succeeded tasks num not equal in job page and job detail page on spark web ui when speculative task(s) exist

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.2
    • Fix Version/s: 2.3.0
    • Component/s: Web UI
    • Labels:
      None

      Description

      when spark.speculation is enabled,and there are some speculative tasks, then we can see succeeded tasks num include speculative tasks on the job page, which however not being included on the job detail page(job stages page).
      When I consider some tasks may run a little slow by the job page's succeeded tasks more than total tasks,which make me want to known which tasks and why,I have to check every stage to find the speculative tasks which is beacause speculative tasks not being included in the stage succeeded task num.
      Can it be improved?

      update two screenshots, succeeded task num is 557 on job page,but 550(by sum) on job detail page(stages),the extra 7 tasks are speculative tasks.

        Attachments

        1. job page.png
          46 kB
          Jinhua Fu
        2. job detail page(stages).png
          69 kB
          Jinhua Fu

          Activity

            People

            • Assignee:
              Jinhua Fu Jinhua Fu
              Reporter:
              Jinhua Fu Jinhua Fu
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: