[HADOOP-4261] Jobs failing in the init stage will never cleanup

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 0.19.0
• Hadoop Flags: Reviewed
• Assignee: Amareshwari Sriramadasu
• Reporter: Amar Kamat
• Created: 24/Sep/08 11:15
• Updated: 08/Jul/09 16:53
• Resolved: 07/Oct/08 07:00

Description

Pre HADOOP-3150, if the job fails in the init stage, job.kill() was called. This used to make sure that the job was cleaned up w.r.t

• staus set to KILLED/FAILED
• job files from the system dir are deleted
• closing of job history files
• making jobtracker aware of this through jobTracker.finalizeJob()
• cleaning up the data structures via JobInProgress.garbageCollect()

Now if the job fails in the init stage, job.fail() is called which doesnt do the cleanup. HADOOP-3150 introduces cleanup tasks which are launched once the job completes i.e killed/failed/succeeded. Jobtracker will never consider this job for scheduling as the job will be in the PREP state forever.

Issue Links

• blocks: HADOOP-4018 limit memory usage in jobtracker