Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7

Reduce progress in the Reducer crosses 1

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      I printed the progress in the Progress.java for every progress made and I found this scenario

      update called for reduce with progress 0.99998355
      update called for reduce with progress 0.99998355
      update called for reduce with progress 1.0000043
      update called for reduce with progress 1.0000043
      update called for reduce with progress 1.0000244
      

      Following are the lines I added to get this

      Progress.java
      /** Called during execution on a leaf node to set its progress. */
        public synchronized void set(float progress) {
          System.out.println("update called for " + status + " with progress " + progress);
          this.progress = progress;
        }
      

      This needs investigation and will also explain why occasionally users see jobs that are complete and their progress less that 100%.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                amar_kamat Amar Kamat
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: