Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: HDFS ACLs (HDFS-4685)
    • Fix Version/s: 2.4.0
    • Component/s: hdfs-client
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Add new methods to AbstractFileSystem and FileContext for manipulating ACLs.

      1. HDFS-5638.patch
        9 kB
        Vinayakumar B
      2. HDFS-5638.patch
        17 kB
        Vinayakumar B
      3. HDFS-5638.patch
        18 kB
        Vinayakumar B
      4. HDFS-5638.2.patch
        7 kB
        Chris Nauroth

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          91d 3h 13m 1 Vinayakumar B 07/Mar/14 10:23
          Patch Available Patch Available Resolved Resolved
          4d 6h 4m 1 Chris Nauroth 11/Mar/14 16:27
          Resolved Resolved Closed Closed
          29d 20h 43m 1 Arun C Murthy 10/Apr/14 14:11
          Allen Wittenauer made changes -
          Fix Version/s 3.0.0 [ 12320356 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1724 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1724/)
          HDFS-5638. HDFS implementation of FileContext API for ACLs. Contributed by Vinayakumar B. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1576405)

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFileContextAcl.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1724 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1724/ ) HDFS-5638 . HDFS implementation of FileContext API for ACLs. Contributed by Vinayakumar B. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1576405 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFileContextAcl.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Hdfs-trunk #1699 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1699/)
          HDFS-5638. HDFS implementation of FileContext API for ACLs. Contributed by Vinayakumar B. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1576405)

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFileContextAcl.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #1699 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1699/ ) HDFS-5638 . HDFS implementation of FileContext API for ACLs. Contributed by Vinayakumar B. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1576405 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFileContextAcl.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #507 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/507/)
          HDFS-5638. HDFS implementation of FileContext API for ACLs. Contributed by Vinayakumar B. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1576405)

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFileContextAcl.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #507 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/507/ ) HDFS-5638 . HDFS implementation of FileContext API for ACLs. Contributed by Vinayakumar B. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1576405 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFileContextAcl.java
          Chris Nauroth made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 3.0.0 [ 12320356 ]
          Fix Version/s 2.4.0 [ 12326143 ]
          Resolution Fixed [ 1 ]
          Hide
          Chris Nauroth added a comment -

          I committed this to trunk, branch-2 and branch-2.4. Thanks again for the contributions, Vinay!

          Show
          Chris Nauroth added a comment - I committed this to trunk, branch-2 and branch-2.4. Thanks again for the contributions, Vinay!
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #5305 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5305/)
          HDFS-5638. HDFS implementation of FileContext API for ACLs. Contributed by Vinayakumar B. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1576405)

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFileContextAcl.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #5305 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5305/ ) HDFS-5638 . HDFS implementation of FileContext API for ACLs. Contributed by Vinayakumar B. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1576405 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFileContextAcl.java
          Hide
          Vinayakumar B added a comment -

          Thanks Chris for review and splitting.

          Show
          Vinayakumar B added a comment - Thanks Chris for review and splitting.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12633816/HDFS-5638.2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6365//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6365//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12633816/HDFS-5638.2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6365//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6365//console This message is automatically generated.
          Chris Nauroth made changes -
          Attachment HDFS-5638.2.patch [ 12633816 ]
          Hide
          Chris Nauroth added a comment -

          Here is a version of the patch containing just the HDFS code, now that I've committed HADOOP-10399. +1 for this patch, pending Jenkins.

          Show
          Chris Nauroth added a comment - Here is a version of the patch containing just the HDFS code, now that I've committed HADOOP-10399 . +1 for this patch, pending Jenkins.
          Chris Nauroth made changes -
          Summary FileContext API for ACLs. HDFS implementation of FileContext API for ACLs.
          Chris Nauroth made changes -
          Link This issue is related to HADOOP-10399 [ HADOOP-10399 ]
          Chris Nauroth made changes -
          Hadoop Flags Reviewed [ 10343 ]
          Target Version/s 3.0.0 [ 12320356 ] 3.0.0, 2.4.0 [ 12320356, 12326143 ]
          Component/s hdfs-client [ 12312928 ]
          Hide
          Chris Nauroth added a comment -

          +1 for the patch. Thanks, Vinay. I like the little delegation trick for reusing the existing tests!

          FYI, I'm going to split off the Hadoop Common portion of the changes into a separate issue and handle that as a separate Jenkins run and commit. You don't need to do anything for this though. I've already +1'd all of the code, and I'll take care of this split.

          Show
          Chris Nauroth added a comment - +1 for the patch. Thanks, Vinay. I like the little delegation trick for reusing the existing tests! FYI, I'm going to split off the Hadoop Common portion of the changes into a separate issue and handle that as a separate Jenkins run and commit. You don't need to do anything for this though. I've already +1'd all of the code, and I'll take care of this split.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12633387/HDFS-5638.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6344//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6344//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12633387/HDFS-5638.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6344//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6344//console This message is automatically generated.
          Vinayakumar B made changes -
          Attachment HDFS-5638.patch [ 12633387 ]
          Hide
          Vinayakumar B added a comment -

          Updated patch with FilterFs failure fix

          Show
          Vinayakumar B added a comment - Updated patch with FilterFs failure fix
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12633345/HDFS-5638.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.fs.TestFilterFs

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6341//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6341//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12633345/HDFS-5638.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.fs.TestFilterFs +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6341//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6341//console This message is automatically generated.
          Vinayakumar B made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Target Version/s HDFS ACLs (HDFS-4685) [ 12325671 ] 3.0.0 [ 12320356 ]
          Vinayakumar B made changes -
          Attachment HDFS-5638.patch [ 12633345 ]
          Hide
          Vinayakumar B added a comment -

          Hi Chris Nauroth,

          Thanks for looking into patch.
          Here is the latest patch with the implentation of ACL methods in Hdfs class Also re-using the existing tests in FsAclBaseTest with FileContext

          Please review and let me know if any further changes required.

          Show
          Vinayakumar B added a comment - Hi Chris Nauroth , Thanks for looking into patch. Here is the latest patch with the implentation of ACL methods in Hdfs class Also re-using the existing tests in FsAclBaseTest with FileContext Please review and let me know if any further changes required.
          Hide
          Chris Nauroth added a comment -

          Hi, Vinayakumar B. This patch makes sense. I was wondering if you ever tried adding the implementation in the Hdfs subclass and testing? Unfortunately, we can only write tests on the HDFS side, not Hadoop Common, because we can't assume the developer's local file system has ACL support.

          Show
          Chris Nauroth added a comment - Hi, Vinayakumar B . This patch makes sense. I was wondering if you ever tried adding the implementation in the Hdfs subclass and testing? Unfortunately, we can only write tests on the HDFS side, not Hadoop Common, because we can't assume the developer's local file system has ACL support.
          Vinayakumar B made changes -
          Assignee Vinay [ vinayrpet ]
          Vinayakumar B made changes -
          Attachment HDFS-5638.patch [ 12618549 ]
          Hide
          Vinayakumar B added a comment -

          Attaching a patch to add ACL apis to FileContext and AbstractFileSystem

          Show
          Vinayakumar B added a comment - Attaching a patch to add ACL apis to FileContext and AbstractFileSystem
          Chris Nauroth made changes -
          Link This issue is related to HDFS-5594 [ HDFS-5594 ]
          Chris Nauroth made changes -
          Component/s namenode [ 12312926 ]
          Component/s hdfs-client [ 12312928 ]
          Component/s security [ 12313400 ]
          Chris Nauroth made changes -
          Field Original Value New Value
          Component/s hdfs-client [ 12312928 ]
          Component/s namenode [ 12312926 ]
          Component/s security [ 12313400 ]
          Chris Nauroth created issue -

            People

            • Assignee:
              Vinayakumar B
              Reporter:
              Chris Nauroth
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development