[MAPREDUCE-463] The job setup and cleanup tasks should be optional

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.21.0
• Hadoop Flags: Reviewed

Release Note

Added Configuration property "mapred.committer.job.setup.cleanup.needed" to specify whether job-setup and job-cleanup is needed for the job output committer. The default value is true.
Added Job.setJobSetupCleanupNeeded and JobContext.getJobSetupCleanupNeeded api for setting/getting the configuration.
If the configuration is set to false, no setup or cleanup will be done.

• Assignee: Amareshwari Sriramadasu
• Reporter: Owen O'Malley
• Created: 08/May/09 07:20
• Updated: 28/Jul/12 00:36
• Resolved: 26/Jun/09 07:53

Description

For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.

Issue Links

• relates to: MAPREDUCE-1099 Setup and cleanup tasks could affect job latency if they are caught running on bad nodes
• relates to: MAPREDUCE-4488 Port MAPREDUCE-463 (The job setup and cleanup tasks should be optional) to branch-1
• relates to: MAPREDUCE-308 A JobInProgressLIstener can change a job without informing other listeners