[MAPREDUCE-4435] Expose JobTracker metrics for number of reducers in shuffle vs. sort vs. reduce phase

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: jobtracker, tasktracker
• Assignee: Unassigned
• Reporter: Eirik Bakke
• Created: 12/Jul/12 22:51
• Updated: 22/Aug/12 07:14

Description

We'd like to be able to show our Cloudera Manager users some more detailed metrics about the number of reducers running at any given time--specifically, how many reducers are running in each of the three possible phases (shuffle, sort, and reduce). This would require the addition of some new overridable methods to the JobTrackerInstrumentation API, plus a little bit of code to actually call them from the JobTracker class. The necessary information seems to already be available in the TaskStatus object. The attached patch (which I've tested on hadoop-common/branch-1.0) shows one way to do it.