[HADOOP-3150] Move task file promotion into the task

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.19.0
• Hadoop Flags: Incompatible change

Release Note

Moved task file promotion to the Task. When the task has finished, it will do a commit and is declared SUCCEDED. Job cleanup is done by a separate task. Job is declared SUCCEDED/FAILED after the cleanup task has finished. Added public classes org.apache.hadoop.mapred.JobContext, TaskAttemptContext, OutputCommitter and FileOutputCommiitter. Added public APIs: public OutputCommitter getOutputCommitter() and
public void setOutputCommitter(Class<? extends OutputCommitter> theClass) in org.apache.hadoop.mapred.JobConf

• Assignee: Amareshwari Sriramadasu
• Reporter: Owen O'Malley
• Created: 01/Apr/08 21:01
• Updated: 08/Jul/09 16:52
• Resolved: 05/Sep/08 10:50

Description

We need to move the task file promotion from the JobTracker to the Task and move it down into the output format.

Issue Links

• is related to: HADOOP-3149 supporting multiple outputs for M/R jobs
• relates to: HADOOP-4100 Scheduler.assignTasks should not be dealing with cleanupTask