Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: HDFS ACLs (HDFS-4685)
    • Fix Version/s: 2.4.0
    • Component/s: hdfs-client
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Add new methods to AbstractFileSystem and FileContext for manipulating ACLs.

      1. HDFS-5638.2.patch
        7 kB
        Chris Nauroth
      2. HDFS-5638.patch
        18 kB
        Vinayakumar B
      3. HDFS-5638.patch
        17 kB
        Vinayakumar B
      4. HDFS-5638.patch
        9 kB
        Vinayakumar B

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1724 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1724/)
          HDFS-5638. HDFS implementation of FileContext API for ACLs. Contributed by Vinayakumar B. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1576405)

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFileContextAcl.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1724 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1724/ ) HDFS-5638 . HDFS implementation of FileContext API for ACLs. Contributed by Vinayakumar B. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1576405 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFileContextAcl.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Hdfs-trunk #1699 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1699/)
          HDFS-5638. HDFS implementation of FileContext API for ACLs. Contributed by Vinayakumar B. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1576405)

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFileContextAcl.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #1699 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1699/ ) HDFS-5638 . HDFS implementation of FileContext API for ACLs. Contributed by Vinayakumar B. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1576405 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFileContextAcl.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #507 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/507/)
          HDFS-5638. HDFS implementation of FileContext API for ACLs. Contributed by Vinayakumar B. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1576405)

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFileContextAcl.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #507 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/507/ ) HDFS-5638 . HDFS implementation of FileContext API for ACLs. Contributed by Vinayakumar B. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1576405 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFileContextAcl.java
          Hide
          Chris Nauroth added a comment -

          I committed this to trunk, branch-2 and branch-2.4. Thanks again for the contributions, Vinay!

          Show
          Chris Nauroth added a comment - I committed this to trunk, branch-2 and branch-2.4. Thanks again for the contributions, Vinay!
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #5305 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5305/)
          HDFS-5638. HDFS implementation of FileContext API for ACLs. Contributed by Vinayakumar B. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1576405)

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFileContextAcl.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #5305 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5305/ ) HDFS-5638 . HDFS implementation of FileContext API for ACLs. Contributed by Vinayakumar B. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1576405 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/fs/Hdfs.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFileContextAcl.java
          Hide
          Vinayakumar B added a comment -

          Thanks Chris for review and splitting.

          Show
          Vinayakumar B added a comment - Thanks Chris for review and splitting.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12633816/HDFS-5638.2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6365//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6365//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12633816/HDFS-5638.2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6365//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6365//console This message is automatically generated.
          Hide
          Chris Nauroth added a comment -

          Here is a version of the patch containing just the HDFS code, now that I've committed HADOOP-10399. +1 for this patch, pending Jenkins.

          Show
          Chris Nauroth added a comment - Here is a version of the patch containing just the HDFS code, now that I've committed HADOOP-10399 . +1 for this patch, pending Jenkins.
          Hide
          Chris Nauroth added a comment -

          +1 for the patch. Thanks, Vinay. I like the little delegation trick for reusing the existing tests!

          FYI, I'm going to split off the Hadoop Common portion of the changes into a separate issue and handle that as a separate Jenkins run and commit. You don't need to do anything for this though. I've already +1'd all of the code, and I'll take care of this split.

          Show
          Chris Nauroth added a comment - +1 for the patch. Thanks, Vinay. I like the little delegation trick for reusing the existing tests! FYI, I'm going to split off the Hadoop Common portion of the changes into a separate issue and handle that as a separate Jenkins run and commit. You don't need to do anything for this though. I've already +1'd all of the code, and I'll take care of this split.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12633387/HDFS-5638.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6344//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6344//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12633387/HDFS-5638.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6344//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6344//console This message is automatically generated.
          Hide
          Vinayakumar B added a comment -

          Updated patch with FilterFs failure fix

          Show
          Vinayakumar B added a comment - Updated patch with FilterFs failure fix
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12633345/HDFS-5638.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.fs.TestFilterFs

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6341//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6341//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12633345/HDFS-5638.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.fs.TestFilterFs +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6341//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6341//console This message is automatically generated.
          Hide
          Vinayakumar B added a comment -

          Hi Chris Nauroth,

          Thanks for looking into patch.
          Here is the latest patch with the implentation of ACL methods in Hdfs class Also re-using the existing tests in FsAclBaseTest with FileContext

          Please review and let me know if any further changes required.

          Show
          Vinayakumar B added a comment - Hi Chris Nauroth , Thanks for looking into patch. Here is the latest patch with the implentation of ACL methods in Hdfs class Also re-using the existing tests in FsAclBaseTest with FileContext Please review and let me know if any further changes required.
          Hide
          Chris Nauroth added a comment -

          Hi, Vinayakumar B. This patch makes sense. I was wondering if you ever tried adding the implementation in the Hdfs subclass and testing? Unfortunately, we can only write tests on the HDFS side, not Hadoop Common, because we can't assume the developer's local file system has ACL support.

          Show
          Chris Nauroth added a comment - Hi, Vinayakumar B . This patch makes sense. I was wondering if you ever tried adding the implementation in the Hdfs subclass and testing? Unfortunately, we can only write tests on the HDFS side, not Hadoop Common, because we can't assume the developer's local file system has ACL support.
          Hide
          Vinayakumar B added a comment -

          Attaching a patch to add ACL apis to FileContext and AbstractFileSystem

          Show
          Vinayakumar B added a comment - Attaching a patch to add ACL apis to FileContext and AbstractFileSystem

            People

            • Assignee:
              Vinayakumar B
              Reporter:
              Chris Nauroth
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development