Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-4062

Speculative attempt scheduling should be aborted when Task has completed

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.10.0, 0.9.3
    • None
    • None

    Description

      In RedundantScheduleTransition (inside TaskImpl), we try to find the oldest running attempt and use it as the causual attempt when doing "addAndScheduleAttempt".

       

      However, the task may have completed at this moment, i.e., the task attempt that was considered running and long-tailed by speculator is now completed. In this case, we may not be able to find any unfinished attempt, which will lead to NPE in following logic (even without NPE, it still makes no sense to proceed with scheduling speculative attempt anyway)

      Attachments

        1. TEZ-4062.001.patch
          3 kB
          Jonathan Turner Eagles

        Issue Links

          Activity

            People

              Chyler Ying Han
              yingdachen Yingda Chen
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 20m
                  1h 20m