Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-5210

Reduce Task Progress shows > 100% when the total size of map outputs (for a single reducer) is high

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      This patch resets the variable totalBytesProcessed before the final merge sothat it will be used for calculating the progress of reducePhase(the 3rd phase of reduce task) correctly.

      Description

      When the total map outputs size (reduce input size) is high, the reported progress is greater than 100%.

        Attachments

        1. HADOOP-5210.patch
          0.6 kB
          Ravi Gummadi
        2. HADOOP-5210.v2.1.patch
          1 kB
          Ravi Gummadi
        3. HADOOP-5210.v2.patch
          0.7 kB
          Ravi Gummadi
        4. HADOOP-5210.v3.patch
          2 kB
          Ravi Gummadi
        5. Picture 3.png
          27 kB
          Jothi Padmanabhan

          Issue Links

            Activity

              People

              • Assignee:
                ravidotg Ravi Gummadi
                Reporter:
                jothipn Jothi Padmanabhan
              • Votes:
                1 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: