Hadoop Common
  1. Hadoop Common
  2. HADOOP-5210

Reduce Task Progress shows > 100% when the total size of map outputs (for a single reducer) is high

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      This patch resets the variable totalBytesProcessed before the final merge sothat it will be used for calculating the progress of reducePhase(the 3rd phase of reduce task) correctly.

      Description

      When the total map outputs size (reduce input size) is high, the reported progress is greater than 100%.

      1. HADOOP-5210.patch
        0.6 kB
        Ravi Gummadi
      2. HADOOP-5210.v2.1.patch
        1 kB
        Ravi Gummadi
      3. HADOOP-5210.v2.patch
        0.7 kB
        Ravi Gummadi
      4. HADOOP-5210.v3.patch
        2 kB
        Ravi Gummadi
      5. Picture 3.png
        27 kB
        Jothi Padmanabhan

        Issue Links

          Activity

          Jeff Hammerbacher made changes -
          Link This issue relates to MAPREDUCE-2264 [ MAPREDUCE-2264 ]
          Devaraj Das made changes -
          Fix Version/s 0.20.1 [ 12313866 ]
          Fix Version/s 0.21.0 [ 12313563 ]
          Devaraj Das made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Fix Version/s 0.21.0 [ 12313563 ]
          Resolution Fixed [ 1 ]
          Ravi Gummadi made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Release Note This patch resets the variable totalBytesProcessed before the final merge sothat it will be used for calculating the progress of reducePhase(the 3rd phase of reduce task) correctly.
          Ravi Gummadi made changes -
          Attachment HADOOP-5210.v3.patch [ 12402622 ]
          Ravi Gummadi made changes -
          Attachment HADOOP-5210.v2.1.patch [ 12402541 ]
          Ravi Gummadi made changes -
          Attachment HADOOP-5210.v2.patch [ 12402260 ]
          Ravi Gummadi made changes -
          Attachment HADOOP-5210.patch [ 12402036 ]
          Ravi Gummadi made changes -
          Assignee Ravi Gummadi [ ravidotg ]
          Amareshwari Sriramadasu made changes -
          Link This issue is related to HADOOP-4000 [ HADOOP-4000 ]
          Jothi Padmanabhan made changes -
          Field Original Value New Value
          Attachment Picture 3.png [ 12399894 ]
          Jothi Padmanabhan created issue -

            People

            • Assignee:
              Ravi Gummadi
              Reporter:
              Jothi Padmanabhan
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development