Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-717

Fix some corner case issues in speculative execution (post hadoop-2141)

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.21.0
    • 0.21.0
    • jobtracker
    • None
    • Fixes some edge cases while using speculative execution

    Description

      Some corner case issues can be fixed:
      1) Setup task should not add anything to the job statistics (since they are really fast and might affect the statistics of a job with few tasks)
      2) The statistics computations should be guarded for cases where things like sumOfSquares could become less than zero (due to rounding errors mostly).
      3) The method TaskInProgress.getCurrentProgressRate() should take into account the COMMIT_PENDING state
      4) The testcase TestSpeculativeExecution.testTaskLATEScheduling could be made more robust

      Attachments

        1. 717.patch
          5 kB
          Devaraj Das
        2. 717.patch
          16 kB
          Devaraj Das

        Activity

          People

            ddas Devaraj Das
            ddas Devaraj Das
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: