Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2289

Permissions race can make getStagingDir fail on local filesystem

    Details

    • Hadoop Flags:
      Reviewed

      Description

      I've observed the following race condition in TestFairSchedulerSystem which uses a MiniMRCluster on top of RawLocalFileSystem:

      • two threads call getStagingDir at the same time
      • Thread A checks fs.exists(stagingArea) and sees false
        • Calls mkdirs(stagingArea, JOB_DIR_PERMISSIONS)
          • mkdirs calls the Java mkdir API which makes the file with umask-based permissions
      • Thread B runs, checks fs.exists(stagingArea) and sees true
        • checks permissions, sees the default permissions, and throws IOE
      • Thread A resumes and sets correct permissions

        Attachments

        1. MAPREDUCE-2289_branch-1.0.patch
          3 kB
          Ahmed Radwan
        2. MAPREDUCE-2289_trunk.patch
          3 kB
          Ahmed Radwan
        3. mapreduce-2289.txt
          3 kB
          Todd Lipcon

          Issue Links

            Activity

              People

              • Assignee:
                ahmed.radwan Ahmed Radwan
                Reporter:
                tlipcon Todd Lipcon
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: