[MAPREDUCE-5154] staging directory deletion fails because delegation tokens have been cancelled

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2.0
• Fix Version/s: 1.2.0
• Component/s: jobtracker
• Assignee: Sandy Ryza
• Reporter: Sandy Ryza
• Created: 17/Apr/13 01:00
• Updated: 15/May/13 05:16
• Resolved: 01/May/13 02:55

Description

In a secure setup, the jobtracker needs the job's delegation tokens to delete the staging directory. MAPREDUCE-4850 made it so that job cleanup staging directory deletion occurs asynchronously, so that it could order it with system directory deletion. This introduced the issue that a job's delegation tokens could be cancelled before the cleanup thread got around to deleting it, causing the deletion to fail.