[HADOOP-3864] JobTracker lockup due to JobInProgress.initTasks taking significant time for large jobs on large clusters

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.18.0
• Fix Version/s: 0.19.0
• Hadoop Flags: Reviewed
• Assignee: Arun Murthy
• Reporter: Arun Murthy
• Created: 30/Jul/08 18:06
• Updated: 08/Jul/09 16:52
• Resolved: 08/Aug/08 23:42

Description

JobInProgress.initTasks takes significant amount of time on a large cluster for large jobs (55k maps * 3 splits), during which the JobInProgress object is locked up.

Simultaneously the JobClient is calling JobTracker.getTaskCompletionEvents which locks the JobTracker & tries to lock the JobInProgress, there-by it starves all heartbeats which are trying to lock the JobTracker - resulting in a lockup.