Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1821

FileContext.createSymlink with kerberos enabled sets wrong owner

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0, 0.23.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      Kerberos enabled on cluster

    • Hadoop Flags:
      Reviewed

      Description

      TEST SETUP
      Using attached sample hdfs java program that illustrates the issue.
      Using cluster with Kerberos enabled on cluster

      1. Compile instructions
        $ javac Symlink.java -cp `hadoop classpath`
        $ jar -cfe Symlink.jar Symlink Symlink.class
      1. create test file for symlink to use
        1. hadoop fs -touchz /user/username/filetest
      1. Create symlink using file context
        2. hadoop jar Symlink.jar ln /user/username/filetest /user/username/testsymlink
      1. Verify owner of test file
        3. hadoop jar Symlink.jar ls /user/username/
        rw-rr- username hdfs /user/jeagles/filetest
        -rwxrwxrwx username@XX.XXXX.XXXXX.XXX hdfs /user/username/testsymlink

      RESULTS
      1. Owner shows 'username@XX.XXXX.XXXXX.XXX' for symlink, expecting 'username'.
      2. Symlink is corrupted and can't removed, since it was created with an invalid user

      ------------------------
      Sample program to create Symlink

      FileContext fc = FileContext.getFileContext(getConf());
      fc.createSymlink(target, symlink, false);

      ---------------------------------------

      1. HDFS-1821-2.patch
        2 kB
        John George
      2. HDFS-1821.patch
        2 kB
        John George

        Activity

        Hide
        John George added a comment -

        [exec] BUILD SUCCESSFUL
        [exec] Total time: 37 seconds
        [exec]
        [exec]
        [exec]
        [exec]
        [exec] +1 overall.
        [exec]
        [exec] +1 @author. The patch does not contain any @author tags.
        [exec]
        [exec] +1 tests included. The patch appears to include 3 new or modified tests.
        [exec]
        [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
        [exec]
        [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
        [exec]
        [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
        [exec]
        [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
        [exec]
        [exec] +1 system test framework. The patch passed system test framework compile.
        [exec]
        [exec]

        Show
        John George added a comment - [exec] BUILD SUCCESSFUL [exec] Total time: 37 seconds [exec] [exec] [exec] [exec] [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system test framework. The patch passed system test framework compile. [exec] [exec]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12475836/HDFS-1821.patch
        against trunk revision 1090357.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these core unit tests:
        org.apache.hadoop.hdfs.TestDFSShell
        org.apache.hadoop.hdfs.TestFileConcurrentReader

        -1 contrib tests. The patch failed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/335//testReport/
        Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/335//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/335//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12475836/HDFS-1821.patch against trunk revision 1090357. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.hdfs.TestDFSShell org.apache.hadoop.hdfs.TestFileConcurrentReader -1 contrib tests. The patch failed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/335//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/335//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/335//console This message is automatically generated.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        +1 patch looks good.

        Show
        Tsz Wo Nicholas Sze added a comment - +1 patch looks good.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        > +1 patch looks good.

        Took a second look,

        • There is another call to getUserName() in NameNode.createSymlink(..). So we need to change it as well.
        • Then, username should be obtained from dirPerms in FSDirectory.addSymlink(..).

        In the existing code, we have FSDirectory.addSymlink(..) calling UserGroupInformation.getCurrentUser() which violates the code design: FSDirectory manages the directory tree data structure and should not care about UserGroupInformation The username resolution should be done in NameNode or FSNamesystem. The use of UserGroupInformation in FSDirectory.addSymlink(..) is the only such violation in FSDirectory.

        John, could you change your patch? Sorry that I didn't see it earlier.

        Show
        Tsz Wo Nicholas Sze added a comment - > +1 patch looks good. Took a second look, There is another call to getUserName() in NameNode.createSymlink(..) . So we need to change it as well. Then, username should be obtained from dirPerms in FSDirectory.addSymlink(..) . In the existing code, we have FSDirectory.addSymlink(..) calling UserGroupInformation.getCurrentUser() which violates the code design: FSDirectory manages the directory tree data structure and should not care about UserGroupInformation The username resolution should be done in NameNode or FSNamesystem . The use of UserGroupInformation in FSDirectory.addSymlink(..) is the only such violation in FSDirectory . John, could you change your patch? Sorry that I didn't see it earlier.
        Hide
        John George added a comment -

        patch to address comments from Nicholas

        Show
        John George added a comment - patch to address comments from Nicholas
        Hide
        John George added a comment -

        [exec]
        [exec] +1 overall.
        [exec]
        [exec] +1 @author. The patch does not contain any @author tags.
        [exec]
        [exec] +1 tests included. The patch appears to include 3 new or modified tests.
        [exec]
        [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
        [exec]
        [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
        [exec]
        [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
        [exec]
        [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
        [exec]
        [exec] +1 system test framework. The patch passed system test framework compile.
        [exec]

        Show
        John George added a comment - [exec] [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system test framework. The patch passed system test framework compile. [exec]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12476347/HDFS-1821-2.patch
        against trunk revision 1091874.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these core unit tests:
        org.apache.hadoop.hdfs.server.datanode.TestBlockReport
        org.apache.hadoop.hdfs.TestFileConcurrentReader

        -1 contrib tests. The patch failed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/367//testReport/
        Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/367//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/367//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12476347/HDFS-1821-2.patch against trunk revision 1091874. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.hdfs.server.datanode.TestBlockReport org.apache.hadoop.hdfs.TestFileConcurrentReader -1 contrib tests. The patch failed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/367//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/367//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/367//console This message is automatically generated.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        +1

        Show
        Tsz Wo Nicholas Sze added a comment - +1
        Hide
        Tsz Wo Nicholas Sze added a comment -

        I have committed this. Thanks, John!

        Show
        Tsz Wo Nicholas Sze added a comment - I have committed this. Thanks, John!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #593 (See https://hudson.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/593/)
        HDFS-1821. Fix username resolution in NameNode.createSymlink(..) and FSDirectory.addSymlink(..). Contributed by John George

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #593 (See https://hudson.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/593/ ) HDFS-1821 . Fix username resolution in NameNode.createSymlink(..) and FSDirectory.addSymlink(..). Contributed by John George
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #643 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/643/)

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #643 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/643/ )
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-22-branch #35 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/35/)

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-22-branch #35 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/35/ )

          People

          • Assignee:
            John George
            Reporter:
            John George
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development