Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 3.0.0, 2.5.0
    • Fix Version/s: 2.5.0
    • Component/s: test
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      We should verify that the XAttr operations adhere to extended ACL permissions.

      In this JIRA we will add a test for this once the XAttr permissions have been refined (HDFS-6556).

      1. HDFS-6603.002.patch
        10 kB
        Stephen Chu
      2. HDFS-6603.001.patch
        11 kB
        Stephen Chu

        Activity

        Hide
        Stephen Chu added a comment -

        This patch adds a test to verify that XAttr operations adhere to ACLs.

        The patch also includes some more minor unit tests:

        • Test renaming a file with XAttr (check that the file keeps attrs)
        • Test removing XAttr from snapshot gets the expected snapshot access control exception
        • Test that NN restart/checkpoint and OfflineImageViewer work when an XAttr with empty value
        Show
        Stephen Chu added a comment - This patch adds a test to verify that XAttr operations adhere to ACLs. The patch also includes some more minor unit tests: Test renaming a file with XAttr (check that the file keeps attrs) Test removing XAttr from snapshot gets the expected snapshot access control exception Test that NN restart/checkpoint and OfflineImageViewer work when an XAttr with empty value
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12653271/HDFS-6603.001.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 5 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.datanode.TestBPOfferService

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/7257//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7257//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12653271/HDFS-6603.001.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 5 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.datanode.TestBPOfferService +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/7257//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7257//console This message is automatically generated.
        Hide
        Stephen Chu added a comment -

        The test failure (TestBPOfferService) above is unrelated to this patch.

        I reran that test locally a few times and it passed.

        Show
        Stephen Chu added a comment - The test failure (TestBPOfferService) above is unrelated to this patch. I reran that test locally a few times and it passed.
        Hide
        Chris Nauroth added a comment -

        Hi, Stephen Chu. Thank you for adding these tests. Do you think it would be better to define testXAttrAcl in FSXAttrBaseTest instead of the subclass TestNameNodeXAttr? That way, we'd exercise the test case through WebHDFS too via TestWebHDFSXAttr. WebHDFS supports the ACL APIs, so I expect this would work fine.

        Show
        Chris Nauroth added a comment - Hi, Stephen Chu . Thank you for adding these tests. Do you think it would be better to define testXAttrAcl in FSXAttrBaseTest instead of the subclass TestNameNodeXAttr ? That way, we'd exercise the test case through WebHDFS too via TestWebHDFSXAttr . WebHDFS supports the ACL APIs, so I expect this would work fine.
        Hide
        Stephen Chu added a comment -

        Thanks, Chris Nauroth! That sounds like a better route. I'll work on moving testXAttrAcl to FSXAttrBaseTest.

        Show
        Stephen Chu added a comment - Thanks, Chris Nauroth ! That sounds like a better route. I'll work on moving testXAttrAcl to FSXAttrBaseTest .
        Hide
        Stephen Chu added a comment -

        Attached a new patch based on Chris's comments.

        I ran the following tests multiple times and they passed successfully:

        • TestNameNodeXAttr
        • TestFSImageWithXAttr
        • TestXAttrWithSnapshot
        • TestOfflineImageViewer
        • TestWebHDFSXAttr
        Show
        Stephen Chu added a comment - Attached a new patch based on Chris's comments. I ran the following tests multiple times and they passed successfully: TestNameNodeXAttr TestFSImageWithXAttr TestXAttrWithSnapshot TestOfflineImageViewer TestWebHDFSXAttr
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12653427/HDFS-6603.002.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 4 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/7265//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7265//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12653427/HDFS-6603.002.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/7265//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7265//console This message is automatically generated.
        Hide
        Stephen Chu added a comment -

        The intermittent test failure above (TestCacheDirectives) is unrelated. I reran TestCacheDirectives locally and it passed.

        Show
        Stephen Chu added a comment - The intermittent test failure above (TestCacheDirectives) is unrelated. I reran TestCacheDirectives locally and it passed.
        Hide
        Chris Nauroth added a comment -

        +1 for the patch. I'll commit this.

        Show
        Chris Nauroth added a comment - +1 for the patch. I'll commit this.
        Hide
        Chris Nauroth added a comment -

        I committed this patch to trunk and branch-2. Stephen, thank you for contributing these tests.

        Show
        Chris Nauroth added a comment - I committed this patch to trunk and branch-2. Stephen, thank you for contributing these tests.
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #5807 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5807/)
        HDFS-6603. Add XAttr with ACL test. Contributed by Stephen Chu. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1607239)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSXAttrBaseTest.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithXAttr.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestXAttrWithSnapshot.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #5807 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5807/ ) HDFS-6603 . Add XAttr with ACL test. Contributed by Stephen Chu. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1607239 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSXAttrBaseTest.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithXAttr.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestXAttrWithSnapshot.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java
        Hide
        Stephen Chu added a comment -

        Thank you, Chris!

        Show
        Stephen Chu added a comment - Thank you, Chris!
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #601 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/601/)
        HDFS-6603. Add XAttr with ACL test. Contributed by Stephen Chu. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1607239)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSXAttrBaseTest.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithXAttr.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestXAttrWithSnapshot.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #601 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/601/ ) HDFS-6603 . Add XAttr with ACL test. Contributed by Stephen Chu. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1607239 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSXAttrBaseTest.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithXAttr.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestXAttrWithSnapshot.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #1819 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1819/)
        HDFS-6603. Add XAttr with ACL test. Contributed by Stephen Chu. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1607239)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSXAttrBaseTest.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithXAttr.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestXAttrWithSnapshot.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1819 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1819/ ) HDFS-6603 . Add XAttr with ACL test. Contributed by Stephen Chu. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1607239 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSXAttrBaseTest.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithXAttr.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestXAttrWithSnapshot.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Hdfs-trunk #1792 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1792/)
        HDFS-6603. Add XAttr with ACL test. Contributed by Stephen Chu. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1607239)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSXAttrBaseTest.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithXAttr.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestXAttrWithSnapshot.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #1792 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1792/ ) HDFS-6603 . Add XAttr with ACL test. Contributed by Stephen Chu. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1607239 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSXAttrBaseTest.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithXAttr.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestXAttrWithSnapshot.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java

          People

          • Assignee:
            Stephen Chu
            Reporter:
            Stephen Chu
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development