Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-717

Fix some corner case issues in speculative execution (post hadoop-2141)

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: jobtracker
    • Labels:
      None
    • Release Note:
      Fixes some edge cases while using speculative execution

      Description

      Some corner case issues can be fixed:
      1) Setup task should not add anything to the job statistics (since they are really fast and might affect the statistics of a job with few tasks)
      2) The statistics computations should be guarded for cases where things like sumOfSquares could become less than zero (due to rounding errors mostly).
      3) The method TaskInProgress.getCurrentProgressRate() should take into account the COMMIT_PENDING state
      4) The testcase TestSpeculativeExecution.testTaskLATEScheduling could be made more robust

        Attachments

        1. 717.patch
          16 kB
          Devaraj Das
        2. 717.patch
          5 kB
          Devaraj Das

          Activity

            People

            • Assignee:
              devaraj Devaraj Das
              Reporter:
              devaraj Devaraj Das
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: