Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-717

Fix some corner case issues in speculative execution (post hadoop-2141)

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: jobtracker
    • Labels:
      None
    • Release Note:
      Fixes some edge cases while using speculative execution

      Description

      Some corner case issues can be fixed:
      1) Setup task should not add anything to the job statistics (since they are really fast and might affect the statistics of a job with few tasks)
      2) The statistics computations should be guarded for cases where things like sumOfSquares could become less than zero (due to rounding errors mostly).
      3) The method TaskInProgress.getCurrentProgressRate() should take into account the COMMIT_PENDING state
      4) The testcase TestSpeculativeExecution.testTaskLATEScheduling could be made more robust

      1. 717.patch
        5 kB
        Devaraj Das
      2. 717.patch
        16 kB
        Devaraj Das

        Activity

        Devaraj Das created issue -
        Devaraj Das made changes -
        Field Original Value New Value
        Attachment 717.patch [ 12412839 ]
        Devaraj Das made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Devaraj Das made changes -
        Attachment 717.patch [ 12413784 ]
        Devaraj Das made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Owen O'Malley made changes -
        Link This issue is blocked by MAPREDUCE-901 [ MAPREDUCE-901 ]
        Owen O'Malley made changes -
        Link This issue is blocked by MAPREDUCE-901 [ MAPREDUCE-901 ]
        Jothi Padmanabhan made changes -
        Release Note Fixes some edge cases while using speculative execution
        Tom White made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Devaraj Das
            Reporter:
            Devaraj Das
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development