[MAPREDUCE-443] snapshot a map-reduce to DFS ... and restore

• Type: New Feature
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Assignee: Owen O'Malley
• Reporter: Eric Baldeschwieler
• Created: 17/Mar/06 12:26
• Updated: 18/Jul/14 17:37
• Resolved: 18/Jul/14 17:36

Description

The idea is to be able to issue a command to the job tracker that
will halt a map-reduce and archive it to a directory in such a way
that it can later be restarted.

We could also set a mode that would cause this to happen to a job
when it fails. This would allow one to debug and restart a failing
job reasonably, which might be important, for long running jobs. It
has certainly been important in similar systems I've seen before. One
could restart with a new jar or work bench a single failing map or reduce.

Issue Links

• is duplicated by: MAPREDUCE-457 Ability to pause/resume tasks
• relates to: HADOOP-313 A stand alone driver for individual tasks
• relates to: MAPREDUCE-460 Should be able to re-run jobs, collecting only missing output
• relates to: MAPREDUCE-452 tasktracker checkpointing capability