[MAPREDUCE-2289] Permissions race can make getStagingDir fail on local filesystem

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.22.0
• Fix Version/s: 1.2.0, 0.23.3, 2.0.2-alpha
• Component/s: job submission
• Hadoop Flags: Reviewed
• Assignee: Ahmed Radwan
• Reporter: Todd Lipcon
• Created: 28/Jan/11 21:04
• Updated: 06/May/13 04:07
• Resolved: 25/Jun/12 21:40

Description

I've observed the following race condition in TestFairSchedulerSystem which uses a MiniMRCluster on top of RawLocalFileSystem:

• two threads call getStagingDir at the same time
• Thread A checks fs.exists(stagingArea) and sees false
  • Calls mkdirs(stagingArea, JOB_DIR_PERMISSIONS)
    • mkdirs calls the Java mkdir API which makes the file with umask-based permissions
• Thread B runs, checks fs.exists(stagingArea) and sees true
  • checks permissions, sees the default permissions, and throws IOE
• Thread A resumes and sets correct permissions

Issue Links

• breaks: HADOOP-7629 regression with MAPREDUCE-2289 - setPermission passed immutable FsPermission (rpc failure)