Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-5210

Reduce Task Progress shows > 100% when the total size of map outputs (for a single reducer) is high

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 0.20.1
    • None
    • None
    • Reviewed
    • This patch resets the variable totalBytesProcessed before the final merge sothat it will be used for calculating the progress of reducePhase(the 3rd phase of reduce task) correctly.

    Description

      When the total map outputs size (reduce input size) is high, the reported progress is greater than 100%.

      Attachments

        1. HADOOP-5210.v3.patch
          2 kB
          Ravi Gummadi
        2. HADOOP-5210.v2.1.patch
          1 kB
          Ravi Gummadi
        3. HADOOP-5210.v2.patch
          0.7 kB
          Ravi Gummadi
        4. HADOOP-5210.patch
          0.6 kB
          Ravi Gummadi
        5. Picture 3.png
          27 kB
          Jothi Padmanabhan

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-7 Reduce progress in the Reducer crosses 1

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-2264 Job status exceeds 100% in some cases

          • Major - Major loss of function.
          • Closed

          Activity

            People

              ravidotg Ravi Gummadi
              jothipn Jothi Padmanabhan
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: