HBase
  1. HBase
  2. HBASE-10239

Improve determinism and debugability of TestAccessController

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.98.0, 0.99.0
    • Fix Version/s: 0.98.0, 0.99.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Separate grant and revoke API invocations to static helper methods in SecureTestUtils. Wait for permissions cache updates using a Predicate. Log the API calls, state checks, and waits.

      1. 10239.patch
        54 kB
        Andrew Purtell
      2. wip-10239.patch
        29 kB
        Andrew Purtell

        Issue Links

          Activity

          Andrew Purtell created issue -
          Andrew Purtell made changes -
          Field Original Value New Value
          Assignee Andrew Purtell [ apurtell ]
          Andrew Purtell made changes -
          Link This issue relates to HBASE-6104 [ HBASE-6104 ]
          Andrew Purtell made changes -
          Description Separate grant and revoke API invocations to static helper methods in SecureUtils (rename to SecureTestUtils). Wait for permissions cache updates using a Predicate. Log the API calls, state checks, and waits. Separate grant and revoke API invocations to static helper methods in SecureTestUtils. Wait for permissions cache updates using a Predicate. Log the API calls, state checks, and waits.
          Hide
          Andrew Purtell added a comment -

          Something like this, attaching work in progress patch. Only updates a few tests, need to move the rest over.

          Show
          Andrew Purtell added a comment - Something like this, attaching work in progress patch. Only updates a few tests, need to move the rest over.
          Andrew Purtell made changes -
          Attachment wip-10239.patch [ 12620536 ]
          Andrew Purtell made changes -
          Attachment wip-10239.patch [ 12620536 ]
          Andrew Purtell made changes -
          Attachment wip-10239.patch [ 12620537 ]
          Andrew Purtell made changes -
          Attachment wip-10239.patch [ 12620537 ]
          Andrew Purtell made changes -
          Attachment wip-10239.patch [ 12620538 ]
          Hide
          Andrew Purtell added a comment -

          Done. Passes JDK6 and 7 locally.

          Show
          Andrew Purtell added a comment - Done. Passes JDK6 and 7 locally.
          Andrew Purtell made changes -
          Attachment 10239.patch [ 12620556 ]
          Andrew Purtell made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Andrew Purtell made changes -
          Link This issue relates to HBASE-6104 [ HBASE-6104 ]
          Andrew Purtell made changes -
          Link This issue blocks HBASE-6104 [ HBASE-6104 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12620556/10239.patch
          against trunk revision .
          ATTACHMENT ID: 12620556

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 11 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop1.1. The patch compiles against the hadoop 1.1 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          -1 site. The patch appears to cause mvn site goal to fail.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/8286//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8286//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8286//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8286//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8286//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8286//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8286//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8286//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8286//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8286//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8286//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12620556/10239.patch against trunk revision . ATTACHMENT ID: 12620556 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 11 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop1.1 . The patch compiles against the hadoop 1.1 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 -1 site . The patch appears to cause mvn site goal to fail. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/8286//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8286//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8286//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8286//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8286//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8286//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8286//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8286//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8286//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8286//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8286//console This message is automatically generated.
          Hide
          ramkrishna.s.vasudevan added a comment -

          Went thro the patch. Looks good to me. +1.
          The updateACL is needed for solving the issue in HBASE-6104?

          Show
          ramkrishna.s.vasudevan added a comment - Went thro the patch. Looks good to me. +1. The updateACL is needed for solving the issue in HBASE-6104 ?
          Hide
          ramkrishna.s.vasudevan added a comment -

          One small concern,
          In the verifyAllowed and verifyDenied we check

          if (obj != null && obj instanceof List<?>) {
          

          But incase of verifyAllowed if the obj itself is null then we silently come out of the method thinking it is success. But it should actually fail because in the verifyAllowed we expect the result.

          Show
          ramkrishna.s.vasudevan added a comment - One small concern, In the verifyAllowed and verifyDenied we check if (obj != null && obj instanceof List<?>) { But incase of verifyAllowed if the obj itself is null then we silently come out of the method thinking it is success. But it should actually fail because in the verifyAllowed we expect the result.
          Hide
          Andrew Purtell added a comment -

          Thanks for the +1 Ram I will commit this a bit later.

          verifyAllowed checks that no exception is thrown and if and only if there is a result collection object it should also not be empty. verifyDenied checks that either an ADE is thrown or if and only if there is a result collection object it should be empty.

          Show
          Andrew Purtell added a comment - Thanks for the +1 Ram I will commit this a bit later. verifyAllowed checks that no exception is thrown and if and only if there is a result collection object it should also not be empty. verifyDenied checks that either an ADE is thrown or if and only if there is a result collection object it should be empty.
          Hide
          Andrew Purtell added a comment -

          Committed to trunk and 0.98

          Show
          Andrew Purtell added a comment - Committed to trunk and 0.98
          Andrew Purtell made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #38 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/38/)
          HBASE-10239. Improve determinism and debugability of TestAccessController (apurtell: rev 1553719)

          • /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          • /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java
          • /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
          • /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestNamespaceCommands.java
          Show
          Hudson added a comment - FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #38 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/38/ ) HBASE-10239 . Improve determinism and debugability of TestAccessController (apurtell: rev 1553719) /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestNamespaceCommands.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in HBase-0.98 #40 (See https://builds.apache.org/job/HBase-0.98/40/)
          HBASE-10239. Improve determinism and debugability of TestAccessController (apurtell: rev 1553719)

          • /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          • /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java
          • /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
          • /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestNamespaceCommands.java
          Show
          Hudson added a comment - FAILURE: Integrated in HBase-0.98 #40 (See https://builds.apache.org/job/HBase-0.98/40/ ) HBASE-10239 . Improve determinism and debugability of TestAccessController (apurtell: rev 1553719) /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestNamespaceCommands.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-1.1 #27 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/27/)
          HBASE-10239. Improve determinism and debugability of TestAccessController (apurtell: rev 1553718)

          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestNamespaceCommands.java
          Show
          Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-1.1 #27 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/27/ ) HBASE-10239 . Improve determinism and debugability of TestAccessController (apurtell: rev 1553718) /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestNamespaceCommands.java
          Hide
          ramkrishna.s.vasudevan added a comment -

          What i meant was I was trying to verify a scenario and added a testcase and was using verifyAllowed to see if the test fails. Actually the test was intended not to return any result.
          But verifyAllowed passed the test. Hence I thought of adding a fail condition even if obj is null.

          Show
          ramkrishna.s.vasudevan added a comment - What i meant was I was trying to verify a scenario and added a testcase and was using verifyAllowed to see if the test fails. Actually the test was intended not to return any result. But verifyAllowed passed the test. Hence I thought of adding a fail condition even if obj is null.
          Hide
          Andrew Purtell added a comment -

          If you want verifyAllowed to fail, then you need to return an empty list or throw an AccessDeniedException.

          Show
          Andrew Purtell added a comment - If you want verifyAllowed to fail, then you need to return an empty list or throw an AccessDeniedException.
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in HBase-TRUNK #4770 (See https://builds.apache.org/job/HBase-TRUNK/4770/)
          HBASE-10239. Improve determinism and debugability of TestAccessController (apurtell: rev 1553718)

          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestNamespaceCommands.java
          Show
          Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK #4770 (See https://builds.apache.org/job/HBase-TRUNK/4770/ ) HBASE-10239 . Improve determinism and debugability of TestAccessController (apurtell: rev 1553718) /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestNamespaceCommands.java
          Hide
          Enis Soztutar added a comment -

          Closing this issue after 0.99.0 release.

          Show
          Enis Soztutar added a comment - Closing this issue after 0.99.0 release.
          Enis Soztutar made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          5h 25m 1 Andrew Purtell 26/Dec/13 22:39
          Patch Available Patch Available Resolved Resolved
          20h 32m 1 Andrew Purtell 27/Dec/13 19:12
          Resolved Resolved Closed Closed
          421d 4h 20m 1 Enis Soztutar 21/Feb/15 23:33

            People

            • Assignee:
              Andrew Purtell
              Reporter:
              Andrew Purtell
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development