[MAPREDUCE-7] Reduce progress in the Reducer crosses 1

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Assignee: Unassigned
• Reporter: Amar Kamat
• Created: 22/Aug/08 09:52
• Updated: 02/May/13 02:29
• Resolved: 07/Oct/09 06:28

Description

I printed the progress in the Progress.java for every progress made and I found this scenario

update called for reduce with progress 0.99998355
update called for reduce with progress 0.99998355
update called for reduce with progress 1.0000043
update called for reduce with progress 1.0000043
update called for reduce with progress 1.0000244

Following are the lines I added to get this

Progress.java

/** Called during execution on a leaf node to set its progress. */
  public synchronized void set(float progress) {
    System.out.println("update called for " + status + " with progress " + progress);
    this.progress = progress;
  }

This needs investigation and will also explain why occasionally users see jobs that are complete and their progress less that 100%.

Issue Links

• is depended upon by: HADOOP-3245 Provide ability to persist running jobs (extend HADOOP-1876)
• is related to: MAPREDUCE-141 reduce % complete incorrect in webui
• relates to: HADOOP-5210 Reduce Task Progress shows > 100% when the total size of map outputs (for a single reducer) is high