[MAPREDUCE-2026] JobTracker.getJobCounters() should not hold JobTracker lock while calling JobInProgress.getCounters()

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.22.0, 0.23.0
• Hadoop Flags: Reviewed
• Assignee: Joydeep Sen Sarma
• Reporter: Scott Chen
• Created: 21/Aug/10 03:40
• Updated: 15/Nov/11 00:48
• Resolved: 03/Feb/11 20:01

Description

JobTracker.getJobCounter() will lock JobTracker and call JobInProgress.getCounters().
JobInProgress.getCounters() can be very expensive because it aggregates all the task counters.
We found that from the JobTracker jstacks that this method is one of the bottleneck of the JobTracker performance.

JobInProgress.getCounters() should be able to be called out side the JobTracker lock because it already has JobInProgress lock.
For example, it is used by jobdetails.jsp without a JobTracker lock.

Issue Links

• duplicates: MAPREDUCE-2114 user finer grained locks in JT getCounters implementation