Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5357

Job staging directory owner checking could fail on Windows

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.1.0-beta, 3.0.0-alpha1
    • Fix Version/s: 2.1.0-beta
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      In JobSubmissionFiles.getStagingDir(), we have following code that will throw exception if the directory owner is not the current user.

            String owner = fsStatus.getOwner();
            if (!(owner.equals(currentUser) || owner.equals(realUser))) {
               throw new IOException("The ownership on the staging directory " +
                            stagingArea + " is not as expected. " +
                            "It is owned by " + owner + ". The directory must " +
                            "be owned by the submitter " + currentUser + " or " +
                            "by " + realUser);
            }
      

      This check will fail on Windows when the underlying file system is LocalFileSystem. Because on Windows, the default file or directory owner could be "Administrators" group if the user belongs to "Administrators" group.

      Quite a few MR unit tests that runs MR mini cluster with localFs as underlying file system fail because of this.

        Attachments

        1. MAPREDUCE-5357-trunk.patch
          0.7 kB
          Chuan Liu

          Issue Links

            Activity

              People

              • Assignee:
                chuanliu Chuan Liu
                Reporter:
                chuanliu Chuan Liu
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: