[MAPREDUCE-3006] MapReduce AM exits prematurely before completely writing and closing the JobHistory file

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.23.0
• Fix Version/s: 0.23.0
• Component/s: applicationmaster, mrv2
• Hadoop Flags: Reviewed
• Assignee: Vinod Kumar Vavilapalli
• Reporter: Vinod Kumar Vavilapalli
• Created: 14/Sep/11 11:08
• Updated: 15/Nov/11 00:50
• Resolved: 18/Sep/11 07:19

Description

Karams was executing a sleep job with 100,000 tasks on a 350 node cluster to test MR AM's scalability and ran into this. The job ran successfully but the history was not available.

I debugged around and figured that the job is finishing prematurely before the JobHistory is written. In most of the cases, we don't see this bug as we have a 5 seconds sleep in AM towards the end.