[MAPREDUCE-3568] Optimize Job's progress calculations in MR AM

• Type: Sub-task
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.23.0
• Fix Version/s: 0.23.1
• Component/s: mr-am, mrv2, performance
• Hadoop Flags: Reviewed
• Release Note: Optimized Job's progress calculations in MR AM.
• Assignee: Vinod Kumar Vavilapalli
• Reporter: Vinod Kumar Vavilapalli
• Created: 15/Dec/11 22:09
• Updated: 05/Mar/12 02:49
• Resolved: 27/Dec/11 19:56

Description

Besides catering to client requests, Job progress is calculated in every heartbeat to the RM so as to print the MR AM's progress. Today the map and reduce progresses are calculated by looking up of each task in a big map while we can simply make do with a scan and aggregate. With large number of tasks, this can make a difference.