[MAPREDUCE-6638] Do not attempt to recover progress from previous job attempts if spill encryption is enabled

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.7.2
• Fix Version/s: 2.9.0, 3.0.0-alpha2
• Component/s: applicationmaster
• Target Version/s: 2.9.0
• Hadoop Flags: Reviewed
• Assignee: Haibo Chen
• Reporter: Karthik Kambatla
• Created: 19/Feb/16 16:28
• Updated: 03/Oct/16 17:42
• Resolved: 03/Oct/16 17:35

Description

Post the fix to CVE-2015-1776, jobs with ecrypted spills enabled cannot be recovered if the AM fails. We should store the key some place safe so they can actually be recovered. If there is no "safe" place, at least we should restart the job by re-running all mappers/reducers.

Issue Links

• is duplicated by: MAPREDUCE-6669 Jobs with encrypted spills don't tolerate AM failures