[HADOOP-4053] Schedulers need to know when a job has completed

• Type: Improvement
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.19.0
• Fix Version/s: 0.19.0
• Hadoop Flags: Reviewed
• Assignee: Amar Kamat
• Reporter: Vivek Ratan
• Created: 02/Sep/08 04:07
• Updated: 08/Jul/09 16:53
• Resolved: 15/Oct/08 22:16

Description

The JobInProgressListener interface is used by the framework to notify Schedulers of when jobs are added, removed, or updated. Right now, there is no way for the Scheduler to know that a job has completed. jobRemoved() is called when a job is retired, which can happen many hours after a job is actually completed. jobUpdated() is called when a job's priority is changed. We need to notify a listener when a job has completed (either successfully, or has failed or been killed).

Issue Links

• blocks: HADOOP-4149 JobQueueJobInProgressListener.jobUpdated() might not work as expected
• blocks: HADOOP-4211 Capacity Scheduler does not divide queue resources properly among users, when jobs are submitted one after other.