Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.92.0
    • Component/s: Coprocessors
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      This is part of an overall implementation of security features for HBase. This issue adds a new AccessController coprocessor which, when enabled, performs authorization checks on all cluster operations, using stored access control lists.

      Description

      Thanks for the clarification Jeff which reminds me to edit this issue.

      Goals of this issue

      1. Client access to HBase is authenticated
      2. User data is private unless access has been granted
      3. Access to data can be granted at a table or per column family basis.

      Non-Goals of this issue

      The following items will be left out of the initial implementation for simplicity:

      1. Row-level or per value (cell) This would require broader changes for storing the ACLs inline with rows. It's still a future goal, but would slow down the initial implementation considerably.
      2. Push down of file ownership to HDFS While table ownership seems like a useful construct to start with (at least to lay the groundwork for future changes), making HBase act as table owners when interacting with HDFS would require more changes. In additional, while HDFS file ownership would make applying quotas easy, and possibly make bulk imports more straightforward, it's not clean it would offer a more secure setup. We'll leave this to evaluate in a later phase.
      3. HBase managed "roles" as collections of permissions We will not model "roles" internally in HBase to begin with. We will instead allow group names to be granted permissions, which will allow some external modeling of roles via group memberships. Groups will be created and manipulated externally to HBase.

      While the assignment of permissions to roles and roles to users (or other roles) allows a great deal of flexibility in security policy, it would add complexity to the initial implementation.

      After the initial implementation, which will appear on this issue, we will evaluate the addition of role definitions internal to HBase in a new JIRA. In this scheme, administrators could assign permissions specifying HDFS groups, and additionally HBase roles. HBase roles would be created and manipulated internally to HBase, and would appear distinct from HDFS groups via some syntactic sugar. HBase role definitions will be allowed to reference other HBase role definitions.

      1. HBASE-3025.1.patch
        90 kB
        Andrew Purtell
      2. HBASE-3025_6.patch
        189 kB
        Gary Helmling
      3. HBASE-3025_5.patch
        190 kB
        Gary Helmling

        Issue Links

          Activity

          Hide
          Andrew Purtell added a comment -

          See HBASE-4990. Destined for the site manual. The piece I have left to do is a capture of an example shell session. I have such a capture but it's led to follow on jiras that need to be resolved for 0.92.1

          Show
          Andrew Purtell added a comment - See HBASE-4990 . Destined for the site manual. The piece I have left to do is a capture of an example shell session. I have such a capture but it's led to follow on jiras that need to be resolved for 0.92.1
          Hide
          Lars Hofhansl added a comment - - edited

          Is there a step-by-step guide somewhere on how to set this up?

          Show
          Lars Hofhansl added a comment - - edited Is there a step-by-step guide somewhere on how to set this up?
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #2459 (See https://builds.apache.org/job/HBase-TRUNK/2459/)
          HBASE-3025 Security: coprocessor based access control

          garyh :
          Files :

          • /hbase/trunk/CHANGES.txt
          • /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access
          • /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java
          • /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
          • /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          • /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java
          • /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java
          • /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java
          • /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java
          • /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java
          • /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java
          • /hbase/trunk/security/src/test/java/org/apache/hadoop/hbase/security/access
          • /hbase/trunk/security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java
          • /hbase/trunk/security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java
          • /hbase/trunk/security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
          • /hbase/trunk/security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java
          • /hbase/trunk/security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          • /hbase/trunk/src/main/resources/hbase-default.xml
          • /hbase/trunk/src/main/ruby/hbase.rb
          • /hbase/trunk/src/main/ruby/hbase/admin.rb
          • /hbase/trunk/src/main/ruby/hbase/hbase.rb
          • /hbase/trunk/src/main/ruby/hbase/security.rb
          • /hbase/trunk/src/main/ruby/shell.rb
          • /hbase/trunk/src/main/ruby/shell/commands.rb
          • /hbase/trunk/src/main/ruby/shell/commands/grant.rb
          • /hbase/trunk/src/main/ruby/shell/commands/revoke.rb
          • /hbase/trunk/src/main/ruby/shell/commands/user_permission.rb
          Show
          Hudson added a comment - Integrated in HBase-TRUNK #2459 (See https://builds.apache.org/job/HBase-TRUNK/2459/ ) HBASE-3025 Security: coprocessor based access control garyh : Files : /hbase/trunk/CHANGES.txt /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java /hbase/trunk/security/src/test/java/org/apache/hadoop/hbase/security/access /hbase/trunk/security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java /hbase/trunk/security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java /hbase/trunk/security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java /hbase/trunk/security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java /hbase/trunk/security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java /hbase/trunk/src/main/resources/hbase-default.xml /hbase/trunk/src/main/ruby/hbase.rb /hbase/trunk/src/main/ruby/hbase/admin.rb /hbase/trunk/src/main/ruby/hbase/hbase.rb /hbase/trunk/src/main/ruby/hbase/security.rb /hbase/trunk/src/main/ruby/shell.rb /hbase/trunk/src/main/ruby/shell/commands.rb /hbase/trunk/src/main/ruby/shell/commands/grant.rb /hbase/trunk/src/main/ruby/shell/commands/revoke.rb /hbase/trunk/src/main/ruby/shell/commands/user_permission.rb
          Hide
          Hudson added a comment -

          Integrated in HBase-0.92 #145 (See https://builds.apache.org/job/HBase-0.92/145/)
          HBASE-3025 Security: coprocessor based access control

          garyh :
          Files :

          • /hbase/branches/0.92/CHANGES.txt
          • /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/access
          • /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java
          • /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
          • /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          • /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java
          • /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java
          • /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java
          • /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java
          • /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java
          • /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java
          • /hbase/branches/0.92/security/src/test/java/org/apache/hadoop/hbase/security/access
          • /hbase/branches/0.92/security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java
          • /hbase/branches/0.92/security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java
          • /hbase/branches/0.92/security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
          • /hbase/branches/0.92/security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java
          • /hbase/branches/0.92/security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java
          • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
          • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java
          • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          • /hbase/branches/0.92/src/main/resources/hbase-default.xml
          • /hbase/branches/0.92/src/main/ruby/hbase.rb
          • /hbase/branches/0.92/src/main/ruby/hbase/admin.rb
          • /hbase/branches/0.92/src/main/ruby/hbase/hbase.rb
          • /hbase/branches/0.92/src/main/ruby/hbase/security.rb
          • /hbase/branches/0.92/src/main/ruby/shell.rb
          • /hbase/branches/0.92/src/main/ruby/shell/commands.rb
          • /hbase/branches/0.92/src/main/ruby/shell/commands/grant.rb
          • /hbase/branches/0.92/src/main/ruby/shell/commands/revoke.rb
          • /hbase/branches/0.92/src/main/ruby/shell/commands/user_permission.rb
          Show
          Hudson added a comment - Integrated in HBase-0.92 #145 (See https://builds.apache.org/job/HBase-0.92/145/ ) HBASE-3025 Security: coprocessor based access control garyh : Files : /hbase/branches/0.92/CHANGES.txt /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/access /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java /hbase/branches/0.92/security/src/test/java/org/apache/hadoop/hbase/security/access /hbase/branches/0.92/security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java /hbase/branches/0.92/security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java /hbase/branches/0.92/security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java /hbase/branches/0.92/security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java /hbase/branches/0.92/security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java /hbase/branches/0.92/src/main/resources/hbase-default.xml /hbase/branches/0.92/src/main/ruby/hbase.rb /hbase/branches/0.92/src/main/ruby/hbase/admin.rb /hbase/branches/0.92/src/main/ruby/hbase/hbase.rb /hbase/branches/0.92/src/main/ruby/hbase/security.rb /hbase/branches/0.92/src/main/ruby/shell.rb /hbase/branches/0.92/src/main/ruby/shell/commands.rb /hbase/branches/0.92/src/main/ruby/shell/commands/grant.rb /hbase/branches/0.92/src/main/ruby/shell/commands/revoke.rb /hbase/branches/0.92/src/main/ruby/shell/commands/user_permission.rb
          Hide
          Gary Helmling added a comment -

          Committed to 0.92 branch and trunk. Thanks again for the reviews.

          Show
          Gary Helmling added a comment - Committed to 0.92 branch and trunk. Thanks again for the reviews.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12504305/HBASE-3025_6.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 10 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/299//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12504305/HBASE-3025_6.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 10 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/299//console This message is automatically generated.
          Hide
          Gary Helmling added a comment -

          Attaching final patch. Only change is removal of a small amount of duplicated code in previous patch from admin.rb.

          Show
          Gary Helmling added a comment - Attaching final patch. Only change is removal of a small amount of duplicated code in previous patch from admin.rb.
          Hide
          stack added a comment -

          You want more reviews Gary?

          Show
          stack added a comment - You want more reviews Gary?
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review3365
          -----------------------------------------------------------

          src/main/ruby/hbase/admin.rb
          <https://reviews.apache.org/r/2041/#comment7545>

          This is duplicated code from a merge somewhere along the way. I'll remove on commit.

          • Gary

          On 2011-11-17 18:48:44, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-17 18:48:44)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844

          src/main/resources/hbase-default.xml 6f98f5d

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 33cd208

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review3365 ----------------------------------------------------------- src/main/ruby/hbase/admin.rb < https://reviews.apache.org/r/2041/#comment7545 > This is duplicated code from a merge somewhere along the way. I'll remove on commit. Gary On 2011-11-17 18:48:44, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-17 18:48:44) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844 src/main/resources/hbase-default.xml 6f98f5d src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 33cd208 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          stack added a comment -

          hadoopqa is failing some tests because of 'too many open files' Its failing distributedlogsplitting for same reason. I'd say its not your patch.

          Show
          stack added a comment - hadoopqa is failing some tests because of 'too many open files' Its failing distributedlogsplitting for same reason. I'd say its not your patch.
          Hide
          Gary Helmling added a comment -

          Hmm, the TestAdmin failure might be due to HConnectionKey changes in HBASE-2742? The error is:

          org.apache.hadoop.hbase.ZooKeeperConnectionException: An error is preventing HBase from connecting to ZooKeeper
          	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:1266)
          	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.setupZookeeperTrackers(HConnectionManager.java:568)
          	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:559)
          	at org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:183)
          	at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:110)
          	at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:1523)
          	at org.apache.hadoop.hbase.client.TestAdmin.testCheckHBaseAvailableClosesConnection(TestAdmin.java:1416)
          ...
          Caused by: java.io.IOException: Too many open files
          	at sun.nio.ch.IOUtil.initPipe(Native Method)
          	at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:49)
          	at sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:18)
          	at java.nio.channels.Selector.open(Selector.java:209)
          	at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:160)
          	at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:331)
          	at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:377)
          	at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.<init>(RecoverableZooKeeper.java:82)
          	at org.apache.hadoop.hbase.zookeeper.ZKUtil.connect(ZKUtil.java:102)
          	at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:131)
          	at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:105)
          	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:1262)
          	... 38 more
          

          Looking into it and will see if I can repro. I've been getting no failures locally.

          Show
          Gary Helmling added a comment - Hmm, the TestAdmin failure might be due to HConnectionKey changes in HBASE-2742 ? The error is: org.apache.hadoop.hbase.ZooKeeperConnectionException: An error is preventing HBase from connecting to ZooKeeper at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:1266) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.setupZookeeperTrackers(HConnectionManager.java:568) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:559) at org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:183) at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:110) at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:1523) at org.apache.hadoop.hbase.client.TestAdmin.testCheckHBaseAvailableClosesConnection(TestAdmin.java:1416) ... Caused by: java.io.IOException: Too many open files at sun.nio.ch.IOUtil.initPipe(Native Method) at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:49) at sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:18) at java.nio.channels.Selector.open(Selector.java:209) at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:160) at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:331) at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:377) at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.<init>(RecoverableZooKeeper.java:82) at org.apache.hadoop.hbase.zookeeper.ZKUtil.connect(ZKUtil.java:102) at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:131) at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:105) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:1262) ... 38 more Looking into it and will see if I can repro. I've been getting no failures locally.
          Hide
          stack added a comment -

          The TestAdmin was because of this patch Gary (The TestShell seems to have come in w/ the HTable changes yesterday of Lars?)

          Show
          stack added a comment - The TestAdmin was because of this patch Gary (The TestShell seems to have come in w/ the HTable changes yesterday of Lars?)
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12504183/HBASE-3025_5.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 10 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 60 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.client.TestAdmin
          org.apache.hadoop.hbase.master.TestDistributedLogSplitting
          org.apache.hadoop.hbase.replication.TestReplication
          org.apache.hadoop.hbase.client.TestShell

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/291//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/291//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/291//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12504183/HBASE-3025_5.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 10 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 60 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestAdmin org.apache.hadoop.hbase.master.TestDistributedLogSplitting org.apache.hadoop.hbase.replication.TestReplication org.apache.hadoop.hbase.client.TestShell Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/291//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/291//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/291//console This message is automatically generated.
          Hide
          Gary Helmling added a comment -

          Latest patch from review board for testing.

          Show
          Gary Helmling added a comment - Latest patch from review board for testing.
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-17 23:10:15, Michael Stack wrote:

          > src/main/ruby/hbase/admin.rb, line 348

          > <https://reviews.apache.org/r/2041/diff/4-5/?file=58485#file58485line348>

          >

          > Did you fix this in another separate patch?

          This just seems like the HBASE-4793 change peeking through when comparing diff 4 and diff 5. Not sure why review board shows it, but it's in trunk, not my patch.

          On 2011-11-17 23:10:15, Michael Stack wrote:

          > src/main/ruby/hbase/security.rb, line 74

          > <https://reviews.apache.org/r/2041/diff/4-5/?file=58487#file58487line74>

          >

          > This seems like pretty important change.

          Yeah, I should have called this out. Wasn't sure if it was already in the previous patch. This is how I stumbled across HBASE-4793 in the first place.

          • Gary

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review3333
          -----------------------------------------------------------

          On 2011-11-17 18:48:44, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-17 18:48:44)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844

          src/main/resources/hbase-default.xml 6f98f5d

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 33cd208

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-17 23:10:15, Michael Stack wrote: > src/main/ruby/hbase/admin.rb, line 348 > < https://reviews.apache.org/r/2041/diff/4-5/?file=58485#file58485line348 > > > Did you fix this in another separate patch? This just seems like the HBASE-4793 change peeking through when comparing diff 4 and diff 5. Not sure why review board shows it, but it's in trunk, not my patch. On 2011-11-17 23:10:15, Michael Stack wrote: > src/main/ruby/hbase/security.rb, line 74 > < https://reviews.apache.org/r/2041/diff/4-5/?file=58487#file58487line74 > > > This seems like pretty important change. Yeah, I should have called this out. Wasn't sure if it was already in the previous patch. This is how I stumbled across HBASE-4793 in the first place. Gary ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review3333 ----------------------------------------------------------- On 2011-11-17 18:48:44, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-17 18:48:44) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844 src/main/resources/hbase-default.xml 6f98f5d src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 33cd208 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review3333
          -----------------------------------------------------------

          Ship it!

          I looked at diff between 4 and 5. Seems fine to me. Some comments below.

          src/main/ruby/hbase/admin.rb
          <https://reviews.apache.org/r/2041/#comment7444>

          Did you fix this in another separate patch?

          src/main/ruby/hbase/security.rb
          <https://reviews.apache.org/r/2041/#comment7445>

          This seems like pretty important change.

          • Michael

          On 2011-11-17 18:48:44, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-17 18:48:44)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844

          src/main/resources/hbase-default.xml 6f98f5d

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 33cd208

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review3333 ----------------------------------------------------------- Ship it! I looked at diff between 4 and 5. Seems fine to me. Some comments below. src/main/ruby/hbase/admin.rb < https://reviews.apache.org/r/2041/#comment7444 > Did you fix this in another separate patch? src/main/ruby/hbase/security.rb < https://reviews.apache.org/r/2041/#comment7445 > This seems like pretty important change. Michael On 2011-11-17 18:48:44, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-17 18:48:44) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844 src/main/resources/hbase-default.xml 6f98f5d src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 33cd208 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/
          -----------------------------------------------------------

          (Updated 2011-11-17 18:48:44.147139)

          Review request for hbase.

          Changes
          -------

          Minor update removing javadoc lines as commented by Stack.

          Summary
          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          • AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.
          • AccessController -
          • implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.
          • implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.
          • ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.
          • Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands
          • Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.
          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs (updated)


          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8
          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844
          src/main/resources/hbase-default.xml 6f98f5d
          src/main/ruby/hbase.rb 4d27191
          src/main/ruby/hbase/admin.rb 33cd208
          src/main/ruby/hbase/hbase.rb beb2450
          src/main/ruby/hbase/security.rb PRE-CREATION
          src/main/ruby/shell.rb 9a47600
          src/main/ruby/shell/commands.rb a352c2e
          src/main/ruby/shell/commands/grant.rb PRE-CREATION
          src/main/ruby/shell/commands/revoke.rb PRE-CREATION
          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing
          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-17 18:48:44.147139) Review request for hbase. Changes ------- Minor update removing javadoc lines as commented by Stack. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. AccessController - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs (updated) security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844 src/main/resources/hbase-default.xml 6f98f5d src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 33cd208 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-15 21:01:17, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java, line 80

          > <https://reviews.apache.org/r/2041/diff/3-4/?file=55450#file55450line80>

          >

          > You didn't want to change name of table?

          Gary Helmling wrote:

          Doesn't seem currently possible to use ".ACL." without changing HTableDescriptor constructors to allow bypassing the isLegalTableName() check? Unless we define a static ACL_TABLEDESC variable in HTD so we can use the protected constructor (same as .META. and ROOT). But doing that seems to violate the separation of concerns to me.

          Michael Stack wrote:

          " But doing that seems to violate the separation of concerns to me."

          This is a 'catalog' table? Non-user table?

          Andrew Purtell wrote:

          The separation that Gary refers to is between security specific implementation detail and the core code.

          OK. I did not get that. I can live w/ this table name.

          • Michael

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review3276
          -----------------------------------------------------------

          On 2011-11-15 19:54:02, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-15 19:54:02)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844

          src/main/resources/hbase-default.xml 6f98f5d

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 17cc891

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-15 21:01:17, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java, line 80 > < https://reviews.apache.org/r/2041/diff/3-4/?file=55450#file55450line80 > > > You didn't want to change name of table? Gary Helmling wrote: Doesn't seem currently possible to use ".ACL." without changing HTableDescriptor constructors to allow bypassing the isLegalTableName() check? Unless we define a static ACL_TABLEDESC variable in HTD so we can use the protected constructor (same as .META. and ROOT ). But doing that seems to violate the separation of concerns to me. Michael Stack wrote: " But doing that seems to violate the separation of concerns to me." This is a 'catalog' table? Non-user table? Andrew Purtell wrote: The separation that Gary refers to is between security specific implementation detail and the core code. OK. I did not get that. I can live w/ this table name. Michael ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review3276 ----------------------------------------------------------- On 2011-11-15 19:54:02, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-15 19:54:02) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844 src/main/resources/hbase-default.xml 6f98f5d src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 17cc891 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-15 21:01:17, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java, line 80

          > <https://reviews.apache.org/r/2041/diff/3-4/?file=55450#file55450line80>

          >

          > You didn't want to change name of table?

          Gary Helmling wrote:

          Doesn't seem currently possible to use ".ACL." without changing HTableDescriptor constructors to allow bypassing the isLegalTableName() check? Unless we define a static ACL_TABLEDESC variable in HTD so we can use the protected constructor (same as .META. and ROOT). But doing that seems to violate the separation of concerns to me.

          Michael Stack wrote:

          " But doing that seems to violate the separation of concerns to me."

          This is a 'catalog' table? Non-user table?

          The separation that Gary refers to is between security specific implementation detail and the core code.

          • Andrew

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review3276
          -----------------------------------------------------------

          On 2011-11-15 19:54:02, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-15 19:54:02)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844

          src/main/resources/hbase-default.xml 6f98f5d

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 17cc891

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-15 21:01:17, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java, line 80 > < https://reviews.apache.org/r/2041/diff/3-4/?file=55450#file55450line80 > > > You didn't want to change name of table? Gary Helmling wrote: Doesn't seem currently possible to use ".ACL." without changing HTableDescriptor constructors to allow bypassing the isLegalTableName() check? Unless we define a static ACL_TABLEDESC variable in HTD so we can use the protected constructor (same as .META. and ROOT ). But doing that seems to violate the separation of concerns to me. Michael Stack wrote: " But doing that seems to violate the separation of concerns to me." This is a 'catalog' table? Non-user table? The separation that Gary refers to is between security specific implementation detail and the core code. Andrew ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review3276 ----------------------------------------------------------- On 2011-11-15 19:54:02, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-15 19:54:02) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844 src/main/resources/hbase-default.xml 6f98f5d src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 17cc891 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-15 21:01:17, Michael Stack wrote:

          > src/main/resources/hbase-default.xml, line 132

          > <https://reviews.apache.org/r/2041/diff/3-4/?file=55466#file55466line132>

          >

          > This looks like its leakage from another issue altogether

          Gary Helmling wrote:

          Ugh, no idea where this came from. Thanks for catching, I'll strip it out.

          Gary Helmling wrote:

          Weird, the section highlighted here is not actually in my patch. Just a trunk change that reviewboard is showing as different from the previous patch uploaded?

          The actual hbase-default.xml change is:

          + <name>zookeeper.znode.acl.parent</name>

          + <value>acl</value>

          + <description>Root ZNode for access control lists.</description>

          + </property>

          +

          + <property>

          No problem. Long as you commit the right stuff (smile).

          On 2011-11-15 21:01:17, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java, line 80

          > <https://reviews.apache.org/r/2041/diff/3-4/?file=55450#file55450line80>

          >

          > You didn't want to change name of table?

          Gary Helmling wrote:

          Doesn't seem currently possible to use ".ACL." without changing HTableDescriptor constructors to allow bypassing the isLegalTableName() check? Unless we define a static ACL_TABLEDESC variable in HTD so we can use the protected constructor (same as .META. and ROOT). But doing that seems to violate the separation of concerns to me.

          " But doing that seems to violate the separation of concerns to me."

          This is a 'catalog' table? Non-user table?

          • Michael

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review3276
          -----------------------------------------------------------

          On 2011-11-15 19:54:02, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-15 19:54:02)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844

          src/main/resources/hbase-default.xml 6f98f5d

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 17cc891

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-15 21:01:17, Michael Stack wrote: > src/main/resources/hbase-default.xml, line 132 > < https://reviews.apache.org/r/2041/diff/3-4/?file=55466#file55466line132 > > > This looks like its leakage from another issue altogether Gary Helmling wrote: Ugh, no idea where this came from. Thanks for catching, I'll strip it out. Gary Helmling wrote: Weird, the section highlighted here is not actually in my patch. Just a trunk change that reviewboard is showing as different from the previous patch uploaded? The actual hbase-default.xml change is: + <name>zookeeper.znode.acl.parent</name> + <value>acl</value> + <description>Root ZNode for access control lists.</description> + </property> + + <property> No problem. Long as you commit the right stuff (smile). On 2011-11-15 21:01:17, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java, line 80 > < https://reviews.apache.org/r/2041/diff/3-4/?file=55450#file55450line80 > > > You didn't want to change name of table? Gary Helmling wrote: Doesn't seem currently possible to use ".ACL." without changing HTableDescriptor constructors to allow bypassing the isLegalTableName() check? Unless we define a static ACL_TABLEDESC variable in HTD so we can use the protected constructor (same as .META. and ROOT ). But doing that seems to violate the separation of concerns to me. " But doing that seems to violate the separation of concerns to me." This is a 'catalog' table? Non-user table? Michael ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review3276 ----------------------------------------------------------- On 2011-11-15 19:54:02, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-15 19:54:02) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844 src/main/resources/hbase-default.xml 6f98f5d src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 17cc891 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-15 21:01:17, Michael Stack wrote:

          > src/main/resources/hbase-default.xml, line 132

          > <https://reviews.apache.org/r/2041/diff/3-4/?file=55466#file55466line132>

          >

          > This looks like its leakage from another issue altogether

          Gary Helmling wrote:

          Ugh, no idea where this came from. Thanks for catching, I'll strip it out.

          Weird, the section highlighted here is not actually in my patch. Just a trunk change that reviewboard is showing as different from the previous patch uploaded?

          The actual hbase-default.xml change is:

          + <name>zookeeper.znode.acl.parent</name>
          + <value>acl</value>
          + <description>Root ZNode for access control lists.</description>
          + </property>
          +
          + <property>

          • Gary

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review3276
          -----------------------------------------------------------

          On 2011-11-15 19:54:02, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-15 19:54:02)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844

          src/main/resources/hbase-default.xml 6f98f5d

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 17cc891

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-15 21:01:17, Michael Stack wrote: > src/main/resources/hbase-default.xml, line 132 > < https://reviews.apache.org/r/2041/diff/3-4/?file=55466#file55466line132 > > > This looks like its leakage from another issue altogether Gary Helmling wrote: Ugh, no idea where this came from. Thanks for catching, I'll strip it out. Weird, the section highlighted here is not actually in my patch. Just a trunk change that reviewboard is showing as different from the previous patch uploaded? The actual hbase-default.xml change is: + <name>zookeeper.znode.acl.parent</name> + <value>acl</value> + <description>Root ZNode for access control lists.</description> + </property> + + <property> Gary ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review3276 ----------------------------------------------------------- On 2011-11-15 19:54:02, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-15 19:54:02) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844 src/main/resources/hbase-default.xml 6f98f5d src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 17cc891 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-15 21:01:17, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java, line 80

          > <https://reviews.apache.org/r/2041/diff/3-4/?file=55450#file55450line80>

          >

          > You didn't want to change name of table?

          Doesn't seem currently possible to use ".ACL." without changing HTableDescriptor constructors to allow bypassing the isLegalTableName() check? Unless we define a static ACL_TABLEDESC variable in HTD so we can use the protected constructor (same as .META. and ROOT). But doing that seems to violate the separation of concerns to me.

          On 2011-11-15 21:01:17, Michael Stack wrote:

          > src/main/resources/hbase-default.xml, line 132

          > <https://reviews.apache.org/r/2041/diff/3-4/?file=55466#file55466line132>

          >

          > This looks like its leakage from another issue altogether

          Ugh, no idea where this came from. Thanks for catching, I'll strip it out.

          • Gary

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review3276
          -----------------------------------------------------------

          On 2011-11-15 19:54:02, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-15 19:54:02)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844

          src/main/resources/hbase-default.xml 6f98f5d

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 17cc891

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-15 21:01:17, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java, line 80 > < https://reviews.apache.org/r/2041/diff/3-4/?file=55450#file55450line80 > > > You didn't want to change name of table? Doesn't seem currently possible to use ".ACL." without changing HTableDescriptor constructors to allow bypassing the isLegalTableName() check? Unless we define a static ACL_TABLEDESC variable in HTD so we can use the protected constructor (same as .META. and ROOT ). But doing that seems to violate the separation of concerns to me. On 2011-11-15 21:01:17, Michael Stack wrote: > src/main/resources/hbase-default.xml, line 132 > < https://reviews.apache.org/r/2041/diff/3-4/?file=55466#file55466line132 > > > This looks like its leakage from another issue altogether Ugh, no idea where this came from. Thanks for catching, I'll strip it out. Gary ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review3276 ----------------------------------------------------------- On 2011-11-15 19:54:02, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-15 19:54:02) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844 src/main/resources/hbase-default.xml 6f98f5d src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 17cc891 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review3276
          -----------------------------------------------------------

          Ship it!

          Patch looks good to go. Some fixups to do on commit below.

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
          <https://reviews.apache.org/r/2041/#comment7327>

          You didn't want to change name of table?

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
          <https://reviews.apache.org/r/2041/#comment7328>

          Thanks

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7330>

          This is easier to understand.

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java
          <https://reviews.apache.org/r/2041/#comment7332>

          Remove this on commit

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java
          <https://reviews.apache.org/r/2041/#comment7331>

          Remove this on commit

          src/main/resources/hbase-default.xml
          <https://reviews.apache.org/r/2041/#comment7333>

          This looks like its leakage from another issue altogether

          • Michael

          On 2011-11-15 19:54:02, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-15 19:54:02)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844

          src/main/resources/hbase-default.xml 6f98f5d

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 17cc891

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review3276 ----------------------------------------------------------- Ship it! Patch looks good to go. Some fixups to do on commit below. security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java < https://reviews.apache.org/r/2041/#comment7327 > You didn't want to change name of table? security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java < https://reviews.apache.org/r/2041/#comment7328 > Thanks security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7330 > This is easier to understand. security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java < https://reviews.apache.org/r/2041/#comment7332 > Remove this on commit security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java < https://reviews.apache.org/r/2041/#comment7331 > Remove this on commit src/main/resources/hbase-default.xml < https://reviews.apache.org/r/2041/#comment7333 > This looks like its leakage from another issue altogether Michael On 2011-11-15 19:54:02, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-15 19:54:02) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844 src/main/resources/hbase-default.xml 6f98f5d src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 17cc891 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-13 19:47:04, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java, line 238

          > <https://reviews.apache.org/r/2041/diff/3/?file=55451#file55451line238>

          >

          > These are expensive calls now that tableinfo has been removed from HRI (IIRC); I don't think there caching going on.

          Gary Helmling wrote:

          I think the HRegionInfo version is the expensive one (doing the HDFS read). For HRegion, this is just returning the HTD instance, so seems like it should be okay...

          OK

          On 2011-11-13 19:47:04, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java, line 41

          > <https://reviews.apache.org/r/2041/diff/3/?file=55453#file55453line41>

          >

          > static?

          Gary Helmling wrote:

          Nested enums are implicitly static

          Thanks.

          • Michael

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review3191
          -----------------------------------------------------------

          On 2011-11-15 19:54:02, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-15 19:54:02)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844

          src/main/resources/hbase-default.xml 6f98f5d

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 17cc891

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-13 19:47:04, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java, line 238 > < https://reviews.apache.org/r/2041/diff/3/?file=55451#file55451line238 > > > These are expensive calls now that tableinfo has been removed from HRI (IIRC); I don't think there caching going on. Gary Helmling wrote: I think the HRegionInfo version is the expensive one (doing the HDFS read). For HRegion, this is just returning the HTD instance, so seems like it should be okay... OK On 2011-11-13 19:47:04, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java, line 41 > < https://reviews.apache.org/r/2041/diff/3/?file=55453#file55453line41 > > > static? Gary Helmling wrote: Nested enums are implicitly static Thanks. Michael ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review3191 ----------------------------------------------------------- On 2011-11-15 19:54:02, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-15 19:54:02) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844 src/main/resources/hbase-default.xml 6f98f5d src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 17cc891 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-13 06:04:14, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java, line 475

          > <https://reviews.apache.org/r/2041/diff/3/?file=55450#file55450line475>

          >

          > Don't have to flush or close the BAOS? It probably does flush/close when you call toByteArray? Over in Writables#getBytes we do something similar to this.

          Andrew Purtell wrote:

          Closing a ByteArrayOutputStream has no effect according to http://download.oracle.com/javase/6/docs/api/java/io/ByteArrayOutputStream.html

          Thanks

          • Michael

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review3187
          -----------------------------------------------------------

          On 2011-11-15 19:54:02, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-15 19:54:02)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844

          src/main/resources/hbase-default.xml 6f98f5d

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 17cc891

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-13 06:04:14, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java, line 475 > < https://reviews.apache.org/r/2041/diff/3/?file=55450#file55450line475 > > > Don't have to flush or close the BAOS? It probably does flush/close when you call toByteArray? Over in Writables#getBytes we do something similar to this. Andrew Purtell wrote: Closing a ByteArrayOutputStream has no effect according to http://download.oracle.com/javase/6/docs/api/java/io/ByteArrayOutputStream.html Thanks Michael ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review3187 ----------------------------------------------------------- On 2011-11-15 19:54:02, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-15 19:54:02) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844 src/main/resources/hbase-default.xml 6f98f5d src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 17cc891 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/
          -----------------------------------------------------------

          (Updated 2011-11-15 19:54:02.538497)

          Review request for hbase.

          Changes
          -------

          Updated patch addressing review comments.

          Summary
          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          • AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.
          • AccessController -
          • implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.
          • implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.
          • ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.
          • Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands
          • Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.
          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs (updated)


          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8
          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844
          src/main/resources/hbase-default.xml 6f98f5d
          src/main/ruby/hbase.rb 4d27191
          src/main/ruby/hbase/admin.rb 17cc891
          src/main/ruby/hbase/hbase.rb beb2450
          src/main/ruby/hbase/security.rb PRE-CREATION
          src/main/ruby/shell.rb 9a47600
          src/main/ruby/shell/commands.rb a352c2e
          src/main/ruby/shell/commands/grant.rb PRE-CREATION
          src/main/ruby/shell/commands/revoke.rb PRE-CREATION
          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing
          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-15 19:54:02.538497) Review request for hbase. Changes ------- Updated patch addressing review comments. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. AccessController - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs (updated) security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 4c5e844 src/main/resources/hbase-default.xml 6f98f5d src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 17cc891 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-13 19:47:04, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java, line 238

          > <https://reviews.apache.org/r/2041/diff/3/?file=55451#file55451line238>

          >

          > These are expensive calls now that tableinfo has been removed from HRI (IIRC); I don't think there caching going on.

          I think the HRegionInfo version is the expensive one (doing the HDFS read). For HRegion, this is just returning the HTD instance, so seems like it should be okay...

          On 2011-11-13 19:47:04, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java, line 344

          > <https://reviews.apache.org/r/2041/diff/3/?file=55451#file55451line344>

          >

          > Is this on every 'action'? Will this become annoying? Loads of spew in logs?

          Yeah, that's going to get pretty noisy. Since we have the audit logging, I think we can just remove this.

          On 2011-11-13 19:47:04, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java, line 498

          > <https://reviews.apache.org/r/2041/diff/3/?file=55451#file55451line498>

          >

          > BaseRegionObserver doesn't noop in this case? Do we need this empty method?

          This is from MasterObserver – need to implement these since java doesn't give us multiple inheritance.

          On 2011-11-13 19:47:04, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java, line 482

          > <https://reviews.apache.org/r/2041/diff/3/?file=55451#file55451line482>

          >

          > So, requirePermission will not find a controlling user – here we are assigning current user as controlling (controlling user has to make the table?)

          "getControllingUser" is a poor name – it's really more the "active user": the remote user in an rpc request, otherwise the currently logged in user for the process. I'll change the name.

          We could use the same method here (I think this code just predates that). I'll update.

          On 2011-11-13 19:47:04, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java, line 676

          > <https://reviews.apache.org/r/2041/diff/3/?file=55451#file55451line676>

          >

          > Oh, so these are independent? You could NOT have perms on family but JUST on an individual cf+qualifier combo?

          Yes.

          On 2011-11-13 19:47:04, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java, line 798

          > <https://reviews.apache.org/r/2041/diff/3/?file=55451#file55451line798>

          >

          > Only controlling user can scan? The person who created the table (or the group I suppose?)

          Anyone with read access can scan (getControllingUser is just poorly named – see above).

          On 2011-11-13 19:47:04, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java, line 865

          > <https://reviews.apache.org/r/2041/diff/3/?file=55451#file55451line865>

          >

          > This code is duplicated. Could be a private method?

          Done.

          On 2011-11-13 19:47:04, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java, line 37

          > <https://reviews.apache.org/r/2041/diff/3/?file=55452#file55452line37>

          >

          > Since it throws exception if fails – and I guess you want to do this because the IOE will have lots of info on the why – then this method might as well be void rather than boolean return?

          Updated grant and revoke to both have void return type (depends on HBASE-4784).

          On 2011-11-13 19:47:04, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java, line 116

          > <https://reviews.apache.org/r/2041/diff/3/?file=55453#file55453line116>

          >

          > Why we have TablePermission pollution in this base Permission class?

          Incomplete refactoring. Fixed.

          On 2011-11-13 19:47:04, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java, line 43

          > <https://reviews.apache.org/r/2041/diff/3/?file=55452#file55452line43>

          >

          > Fix this first sentence... reads wrong?

          Done.

          On 2011-11-13 19:47:04, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java, line 651

          > <https://reviews.apache.org/r/2041/diff/3/?file=55451#file55451line651>

          >

          > This is not a good name for a boolean. Shouldn't it be aclRegion? isAclRegion is the name of the method that checks the boolean is true.

          Changed to aclRegion.

          On 2011-11-13 19:47:04, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java, line 41

          > <https://reviews.apache.org/r/2041/diff/3/?file=55453#file55453line41>

          >

          > static?

          Nested enums are implicitly static

          On 2011-11-13 19:47:04, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java, line 94

          > <https://reviews.apache.org/r/2041/diff/3/?file=55453#file55453line94>

          >

          > You need this? Else its a read-only class. Just create new instance if want to change Actions?

          Yeah, good point. This was only used in tests, which were easily fixed, so I've removed it. The class is now immutable (except for writable).

          On 2011-11-13 19:47:04, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java, line 95

          > <https://reviews.apache.org/r/2041/diff/3/?file=55456#file55456line95>

          >

          > Do we need a hash method? Same for other Permission objects?

          Added hashCode() for Permission, TablePermission, and UserPermission

          • Gary

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review3191
          -----------------------------------------------------------

          On 2011-11-01 21:18:27, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-01 21:18:27)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          src/main/resources/hbase-default.xml 3785533

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 61e04d8

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-13 19:47:04, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java, line 238 > < https://reviews.apache.org/r/2041/diff/3/?file=55451#file55451line238 > > > These are expensive calls now that tableinfo has been removed from HRI (IIRC); I don't think there caching going on. I think the HRegionInfo version is the expensive one (doing the HDFS read). For HRegion, this is just returning the HTD instance, so seems like it should be okay... On 2011-11-13 19:47:04, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java, line 344 > < https://reviews.apache.org/r/2041/diff/3/?file=55451#file55451line344 > > > Is this on every 'action'? Will this become annoying? Loads of spew in logs? Yeah, that's going to get pretty noisy. Since we have the audit logging, I think we can just remove this. On 2011-11-13 19:47:04, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java, line 498 > < https://reviews.apache.org/r/2041/diff/3/?file=55451#file55451line498 > > > BaseRegionObserver doesn't noop in this case? Do we need this empty method? This is from MasterObserver – need to implement these since java doesn't give us multiple inheritance. On 2011-11-13 19:47:04, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java, line 482 > < https://reviews.apache.org/r/2041/diff/3/?file=55451#file55451line482 > > > So, requirePermission will not find a controlling user – here we are assigning current user as controlling (controlling user has to make the table?) "getControllingUser" is a poor name – it's really more the "active user": the remote user in an rpc request, otherwise the currently logged in user for the process. I'll change the name. We could use the same method here (I think this code just predates that). I'll update. On 2011-11-13 19:47:04, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java, line 676 > < https://reviews.apache.org/r/2041/diff/3/?file=55451#file55451line676 > > > Oh, so these are independent? You could NOT have perms on family but JUST on an individual cf+qualifier combo? Yes. On 2011-11-13 19:47:04, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java, line 798 > < https://reviews.apache.org/r/2041/diff/3/?file=55451#file55451line798 > > > Only controlling user can scan? The person who created the table (or the group I suppose?) Anyone with read access can scan (getControllingUser is just poorly named – see above). On 2011-11-13 19:47:04, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java, line 865 > < https://reviews.apache.org/r/2041/diff/3/?file=55451#file55451line865 > > > This code is duplicated. Could be a private method? Done. On 2011-11-13 19:47:04, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java, line 37 > < https://reviews.apache.org/r/2041/diff/3/?file=55452#file55452line37 > > > Since it throws exception if fails – and I guess you want to do this because the IOE will have lots of info on the why – then this method might as well be void rather than boolean return? Updated grant and revoke to both have void return type (depends on HBASE-4784 ). On 2011-11-13 19:47:04, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java, line 116 > < https://reviews.apache.org/r/2041/diff/3/?file=55453#file55453line116 > > > Why we have TablePermission pollution in this base Permission class? Incomplete refactoring. Fixed. On 2011-11-13 19:47:04, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java, line 43 > < https://reviews.apache.org/r/2041/diff/3/?file=55452#file55452line43 > > > Fix this first sentence... reads wrong? Done. On 2011-11-13 19:47:04, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java, line 651 > < https://reviews.apache.org/r/2041/diff/3/?file=55451#file55451line651 > > > This is not a good name for a boolean. Shouldn't it be aclRegion? isAclRegion is the name of the method that checks the boolean is true. Changed to aclRegion. On 2011-11-13 19:47:04, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java, line 41 > < https://reviews.apache.org/r/2041/diff/3/?file=55453#file55453line41 > > > static? Nested enums are implicitly static On 2011-11-13 19:47:04, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java, line 94 > < https://reviews.apache.org/r/2041/diff/3/?file=55453#file55453line94 > > > You need this? Else its a read-only class. Just create new instance if want to change Actions? Yeah, good point. This was only used in tests, which were easily fixed, so I've removed it. The class is now immutable (except for writable). On 2011-11-13 19:47:04, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java, line 95 > < https://reviews.apache.org/r/2041/diff/3/?file=55456#file55456line95 > > > Do we need a hash method? Same for other Permission objects? Added hashCode() for Permission, TablePermission, and UserPermission Gary ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review3191 ----------------------------------------------------------- On 2011-11-01 21:18:27, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-01 21:18:27) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/resources/hbase-default.xml 3785533 src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 61e04d8 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review3197
          -----------------------------------------------------------

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7093>

          HRegion was returned from e.getRegion().
          The getTableDesc() returns HTD directly.
          public HTableDescriptor getTableDesc()

          { return this.htableDescriptor; }
          • Ted

          On 2011-11-01 21:18:27, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-01 21:18:27)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          src/main/resources/hbase-default.xml 3785533

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 61e04d8

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review3197 ----------------------------------------------------------- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7093 > HRegion was returned from e.getRegion(). The getTableDesc() returns HTD directly. public HTableDescriptor getTableDesc() { return this.htableDescriptor; } Ted On 2011-11-01 21:18:27, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-01 21:18:27) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/resources/hbase-default.xml 3785533 src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 61e04d8 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-13 06:04:14, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java, line 214

          > <https://reviews.apache.org/r/2041/diff/3/?file=55451#file55451line214>

          >

          > What happens now? Table is wide open for access?

          Not necessarily, if previous permissions were applied successfully, the previous permissions are still up in the ZK mirror and active across the cluster. However, it is an unresolved matter what should be done here. Take the table offline is one option, essentially forcing a "reboot" of access control across the cluster for the table in question. Would be disruptive.

          On 2011-11-13 06:04:14, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java, line 209

          > <https://reviews.apache.org/r/2041/diff/3/?file=55451#file55451line209>

          >

          > Its a pain isn't it that we are using Writables serializing to znodes. It'd be more human friendly doing json... but that we can do later.

          Concur

          On 2011-11-13 06:04:14, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java, line 475

          > <https://reviews.apache.org/r/2041/diff/3/?file=55450#file55450line475>

          >

          > Don't have to flush or close the BAOS? It probably does flush/close when you call toByteArray? Over in Writables#getBytes we do something similar to this.

          Closing a ByteArrayOutputStream has no effect according to http://download.oracle.com/javase/6/docs/api/java/io/ByteArrayOutputStream.html

          On 2011-11-13 06:04:14, Michael Stack wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java, line 427

          > <https://reviews.apache.org/r/2041/diff/3/?file=55450#file55450line427>

          >

          > This is a bit of a pain having to make a String of it then going back to byte arrays after parse. It'll do... Maybe add a TODO: about making this more efficient.

          This is some friction we see in several places in HBase code. byte[] is a really generic and efficient representation, but String contains many useful methods that Byte (or Bytes) lack.

          • Andrew

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review3187
          -----------------------------------------------------------

          On 2011-11-01 21:18:27, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-01 21:18:27)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          src/main/resources/hbase-default.xml 3785533

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 61e04d8

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-13 06:04:14, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java, line 214 > < https://reviews.apache.org/r/2041/diff/3/?file=55451#file55451line214 > > > What happens now? Table is wide open for access? Not necessarily, if previous permissions were applied successfully, the previous permissions are still up in the ZK mirror and active across the cluster. However, it is an unresolved matter what should be done here. Take the table offline is one option, essentially forcing a "reboot" of access control across the cluster for the table in question. Would be disruptive. On 2011-11-13 06:04:14, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java, line 209 > < https://reviews.apache.org/r/2041/diff/3/?file=55451#file55451line209 > > > Its a pain isn't it that we are using Writables serializing to znodes. It'd be more human friendly doing json... but that we can do later. Concur On 2011-11-13 06:04:14, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java, line 475 > < https://reviews.apache.org/r/2041/diff/3/?file=55450#file55450line475 > > > Don't have to flush or close the BAOS? It probably does flush/close when you call toByteArray? Over in Writables#getBytes we do something similar to this. Closing a ByteArrayOutputStream has no effect according to http://download.oracle.com/javase/6/docs/api/java/io/ByteArrayOutputStream.html On 2011-11-13 06:04:14, Michael Stack wrote: > security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java, line 427 > < https://reviews.apache.org/r/2041/diff/3/?file=55450#file55450line427 > > > This is a bit of a pain having to make a String of it then going back to byte arrays after parse. It'll do... Maybe add a TODO: about making this more efficient. This is some friction we see in several places in HBase code. byte[] is a really generic and efficient representation, but String contains many useful methods that Byte (or Bytes) lack. Andrew ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review3187 ----------------------------------------------------------- On 2011-11-01 21:18:27, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-01 21:18:27) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/resources/hbase-default.xml 3785533 src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 61e04d8 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-13 19:47:24, Michael Stack wrote:

          > You running this at TM G?

          An earlier iteration of this code is in production. AccessController is essentially the same. The plan is to upstream 3025, the RPC changes, and secure ZooKeeper integration, then pull the resulting 0.92 release back in to a future production environment.

          • Andrew

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review3192
          -----------------------------------------------------------

          On 2011-11-01 21:18:27, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-01 21:18:27)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          src/main/resources/hbase-default.xml 3785533

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 61e04d8

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-13 19:47:24, Michael Stack wrote: > You running this at TM G? An earlier iteration of this code is in production. AccessController is essentially the same. The plan is to upstream 3025, the RPC changes, and secure ZooKeeper integration, then pull the resulting 0.92 release back in to a future production environment. Andrew ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review3192 ----------------------------------------------------------- On 2011-11-01 21:18:27, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-01 21:18:27) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/resources/hbase-default.xml 3785533 src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 61e04d8 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review3192
          -----------------------------------------------------------

          You running this at TM G?

          • Michael

          On 2011-11-01 21:18:27, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-01 21:18:27)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          src/main/resources/hbase-default.xml 3785533

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 61e04d8

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review3192 ----------------------------------------------------------- You running this at TM G? Michael On 2011-11-01 21:18:27, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-01 21:18:27) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/resources/hbase-default.xml 3785533 src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 61e04d8 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review3191
          -----------------------------------------------------------

          This is great.

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7055>

          These are expensive calls now that tableinfo has been removed from HRI (IIRC); I don't think there caching going on.

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7056>

          I like the way this is done; a data structure w/ all the detail on why the fail or pass

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7057>

          Is this on every 'action'? Will this become annoying? Loads of spew in logs?

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7058>

          Good; good feedback on why the access failed

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7059>

          You might in future do this method comment as the javadoc on the @return param

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7061>

          So, requirePermission will not find a controlling user – here we are assigning current user as controlling (controlling user has to make the table?)

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7062>

          BaseRegionObserver doesn't noop in this case? Do we need this empty method?

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7063>

          This is not a good name for a boolean. Shouldn't it be aclRegion? isAclRegion is the name of the method that checks the boolean is true.

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7064>

          Oh, so these are independent? You could NOT have perms on family but JUST on an individual cf+qualifier combo?

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7065>

          Good.

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7066>

          Only controlling user can scan? The person who created the table (or the group I suppose?)

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7067>

          This code is duplicated. Could be a private method?

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7068>

          Nice (an audit log)

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java
          <https://reviews.apache.org/r/2041/#comment7069>

          Since it throws exception if fails – and I guess you want to do this because the IOE will have lots of info on the why – then this method might as well be void rather than boolean return?

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java
          <https://reviews.apache.org/r/2041/#comment7070>

          Fix this first sentence... reads wrong?

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java
          <https://reviews.apache.org/r/2041/#comment7071>

          Nice documentation

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java
          <https://reviews.apache.org/r/2041/#comment7072>

          static?

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java
          <https://reviews.apache.org/r/2041/#comment7073>

          You need this? Else its a read-only class. Just create new instance if want to change Actions?

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java
          <https://reviews.apache.org/r/2041/#comment7074>

          Why we have TablePermission pollution in this base Permission class?

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java
          <https://reviews.apache.org/r/2041/#comment7075>

          Do we need a hash method? Same for other Permission objects?

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java
          <https://reviews.apache.org/r/2041/#comment7076>

          You have to have the 'access' subpackage under security? What else beside this access stuff would be in the package? (just wondering if we are down too deep in the packaging – if we could come up one).

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
          <https://reviews.apache.org/r/2041/#comment7077>

          Could you put all these tests into one test class so only one cluster spin-up to test access? If too hard, don't worry... just asking.

          Or maybe not... it looks like you are getting good use out of your cluster spinup in this test – nice test.

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
          <https://reviews.apache.org/r/2041/#comment7078>

          Great test. Comprehensive.

          • Michael

          On 2011-11-01 21:18:27, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-01 21:18:27)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          src/main/resources/hbase-default.xml 3785533

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 61e04d8

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review3191 ----------------------------------------------------------- This is great. security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7055 > These are expensive calls now that tableinfo has been removed from HRI (IIRC); I don't think there caching going on. security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7056 > I like the way this is done; a data structure w/ all the detail on why the fail or pass security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7057 > Is this on every 'action'? Will this become annoying? Loads of spew in logs? security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7058 > Good; good feedback on why the access failed security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7059 > You might in future do this method comment as the javadoc on the @return param security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7061 > So, requirePermission will not find a controlling user – here we are assigning current user as controlling (controlling user has to make the table?) security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7062 > BaseRegionObserver doesn't noop in this case? Do we need this empty method? security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7063 > This is not a good name for a boolean. Shouldn't it be aclRegion? isAclRegion is the name of the method that checks the boolean is true. security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7064 > Oh, so these are independent? You could NOT have perms on family but JUST on an individual cf+qualifier combo? security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7065 > Good. security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7066 > Only controlling user can scan? The person who created the table (or the group I suppose?) security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7067 > This code is duplicated. Could be a private method? security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7068 > Nice (an audit log) security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java < https://reviews.apache.org/r/2041/#comment7069 > Since it throws exception if fails – and I guess you want to do this because the IOE will have lots of info on the why – then this method might as well be void rather than boolean return? security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java < https://reviews.apache.org/r/2041/#comment7070 > Fix this first sentence... reads wrong? security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java < https://reviews.apache.org/r/2041/#comment7071 > Nice documentation security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java < https://reviews.apache.org/r/2041/#comment7072 > static? security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java < https://reviews.apache.org/r/2041/#comment7073 > You need this? Else its a read-only class. Just create new instance if want to change Actions? security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java < https://reviews.apache.org/r/2041/#comment7074 > Why we have TablePermission pollution in this base Permission class? security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java < https://reviews.apache.org/r/2041/#comment7075 > Do we need a hash method? Same for other Permission objects? security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java < https://reviews.apache.org/r/2041/#comment7076 > You have to have the 'access' subpackage under security? What else beside this access stuff would be in the package? (just wondering if we are down too deep in the packaging – if we could come up one). security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java < https://reviews.apache.org/r/2041/#comment7077 > Could you put all these tests into one test class so only one cluster spin-up to test access? If too hard, don't worry... just asking. Or maybe not... it looks like you are getting good use out of your cluster spinup in this test – nice test. security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java < https://reviews.apache.org/r/2041/#comment7078 > Great test. Comprehensive. Michael On 2011-11-01 21:18:27, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-01 21:18:27) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/resources/hbase-default.xml 3785533 src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 61e04d8 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          Andrew Purtell added a comment -

          I'll sort out the disposition of this and the other two patches with Gary next week. With any luck we'll get them committed next week as well.

          Show
          Andrew Purtell added a comment - I'll sort out the disposition of this and the other two patches with Gary next week. With any luck we'll get them committed next week as well.
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review3187
          -----------------------------------------------------------

          About 1/4 way through. Will pick up again in morning.

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java
          <https://reviews.apache.org/r/2041/#comment7008>

          Is it on whenever I'm doing access control?

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java
          <https://reviews.apache.org/r/2041/#comment7009>

          Interesting

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
          <https://reviews.apache.org/r/2041/#comment7010>

          Is acl a good name? I hate ROOT and then .META. Its dumb. All catalog tables should look the same. ROOT will likely go away soon. That would tend to rule the name be .ACL. But then leading off w/ a dot is a bit of a pain especially when you copy it local filesystem (it won't show in listings). On other hand, maybe thats ok... makes it special. And our special dirs tend to lead off with a '.' as in '.logs'., etc.

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
          <https://reviews.apache.org/r/2041/#comment7011>

          We ensure a user can't have a '@" prefix I presume (haven't read all the code yet).

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
          <https://reviews.apache.org/r/2041/#comment7012>

          This is great. Nice. Clean. What happens if we ever want to do cell-level?

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
          <https://reviews.apache.org/r/2041/#comment7013>

          I'd suggest you set an example with this new table and instead of having the cf be 'info', instead have it be 'l' as short for lists (you are giving an example by having short cf names).

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
          <https://reviews.apache.org/r/2041/#comment7014>

          If no family qualifier, we are granting perm on whole table? Thats ok?

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
          <https://reviews.apache.org/r/2041/#comment7015>

          Should we check family is not null? Doesn't a qualifier have to have a family to qualify? This should be inside the family check?

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
          <https://reviews.apache.org/r/2041/#comment7016>

          Ok good.

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
          <https://reviews.apache.org/r/2041/#comment7017>

          Usually space either side of operators as in 'int i = 0' rather than 'int i=0'. Etc.

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
          <https://reviews.apache.org/r/2041/#comment7018>

          Oh, a byte per action? Thats grand.

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
          <https://reviews.apache.org/r/2041/#comment7019>

          Should it throw exception?

          Should we read for an ACL first and not write a delete if none present (throwing exception if nothing to delete)?

          I think I know why no effect

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
          <https://reviews.apache.org/r/2041/#comment7020>

          When I do this? And what happens if perms are edited subsequently? Are they considered?

          Or is this method for testing like the one that follows?

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
          <https://reviews.apache.org/r/2041/#comment7021>

          We're doing the ACL table, not .META. This a stale comment?

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
          <https://reviews.apache.org/r/2041/#comment7022>

          Stale comment?

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
          <https://reviews.apache.org/r/2041/#comment7023>

          Just point at class comment rather than dup it here?

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
          <https://reviews.apache.org/r/2041/#comment7024>

          Probably have to do that over in core master classes. Its there we are guaranteeing root up before meta...

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
          <https://reviews.apache.org/r/2041/#comment7025>

          This is a bit of a pain having to make a String of it then going back to byte arrays after parse. It'll do... Maybe add a TODO: about making this more efficient.

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
          <https://reviews.apache.org/r/2041/#comment7026>

          Strings?

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
          <https://reviews.apache.org/r/2041/#comment7027>

          Don't have to flush or close the BAOS? It probably does flush/close when you call toByteArray? Over in Writables#getBytes we do something similar to this.

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7028>

          Sweet

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7029>

          These should all be finals? This an immutable data structure?

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7030>

          What is diff between toContextString and toString?

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7031>

          Whats an aclregion?

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7032>

          We load on start of the controller? Then if access is subsequently edited, its done via the controller only? It updates its internal state as well as the acl table?

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7034>

          ok

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7035>

          Its a pain isn't it that we are using Writables serializing to znodes. It'd be more human friendly doing json... but that we can do later.

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
          <https://reviews.apache.org/r/2041/#comment7036>

          What happens now? Table is wide open for access?

          • Michael

          On 2011-11-01 21:18:27, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-01 21:18:27)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          src/main/resources/hbase-default.xml 3785533

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 61e04d8

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review3187 ----------------------------------------------------------- About 1/4 way through. Will pick up again in morning. security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java < https://reviews.apache.org/r/2041/#comment7008 > Is it on whenever I'm doing access control? security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java < https://reviews.apache.org/r/2041/#comment7009 > Interesting security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java < https://reviews.apache.org/r/2041/#comment7010 > Is acl a good name? I hate ROOT and then .META. Its dumb. All catalog tables should look the same. ROOT will likely go away soon. That would tend to rule the name be .ACL. But then leading off w/ a dot is a bit of a pain especially when you copy it local filesystem (it won't show in listings). On other hand, maybe thats ok... makes it special. And our special dirs tend to lead off with a '.' as in '.logs'., etc. security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java < https://reviews.apache.org/r/2041/#comment7011 > We ensure a user can't have a '@" prefix I presume (haven't read all the code yet). security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java < https://reviews.apache.org/r/2041/#comment7012 > This is great. Nice. Clean. What happens if we ever want to do cell-level? security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java < https://reviews.apache.org/r/2041/#comment7013 > I'd suggest you set an example with this new table and instead of having the cf be 'info', instead have it be 'l' as short for lists (you are giving an example by having short cf names). security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java < https://reviews.apache.org/r/2041/#comment7014 > If no family qualifier, we are granting perm on whole table? Thats ok? security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java < https://reviews.apache.org/r/2041/#comment7015 > Should we check family is not null? Doesn't a qualifier have to have a family to qualify? This should be inside the family check? security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java < https://reviews.apache.org/r/2041/#comment7016 > Ok good. security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java < https://reviews.apache.org/r/2041/#comment7017 > Usually space either side of operators as in 'int i = 0' rather than 'int i=0'. Etc. security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java < https://reviews.apache.org/r/2041/#comment7018 > Oh, a byte per action? Thats grand. security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java < https://reviews.apache.org/r/2041/#comment7019 > Should it throw exception? Should we read for an ACL first and not write a delete if none present (throwing exception if nothing to delete)? I think I know why no effect security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java < https://reviews.apache.org/r/2041/#comment7020 > When I do this? And what happens if perms are edited subsequently? Are they considered? Or is this method for testing like the one that follows? security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java < https://reviews.apache.org/r/2041/#comment7021 > We're doing the ACL table, not .META. This a stale comment? security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java < https://reviews.apache.org/r/2041/#comment7022 > Stale comment? security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java < https://reviews.apache.org/r/2041/#comment7023 > Just point at class comment rather than dup it here? security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java < https://reviews.apache.org/r/2041/#comment7024 > Probably have to do that over in core master classes. Its there we are guaranteeing root up before meta... security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java < https://reviews.apache.org/r/2041/#comment7025 > This is a bit of a pain having to make a String of it then going back to byte arrays after parse. It'll do... Maybe add a TODO: about making this more efficient. security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java < https://reviews.apache.org/r/2041/#comment7026 > Strings? security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java < https://reviews.apache.org/r/2041/#comment7027 > Don't have to flush or close the BAOS? It probably does flush/close when you call toByteArray? Over in Writables#getBytes we do something similar to this. security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7028 > Sweet security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7029 > These should all be finals? This an immutable data structure? security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7030 > What is diff between toContextString and toString? security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7031 > Whats an aclregion? security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7032 > We load on start of the controller? Then if access is subsequently edited, its done via the controller only? It updates its internal state as well as the acl table? security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7034 > ok security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7035 > Its a pain isn't it that we are using Writables serializing to znodes. It'd be more human friendly doing json... but that we can do later. security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java < https://reviews.apache.org/r/2041/#comment7036 > What happens now? Table is wide open for access? Michael On 2011-11-01 21:18:27, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-01 21:18:27) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/resources/hbase-default.xml 3785533 src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 61e04d8 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          stack added a comment -

          This is in way of 0.92 release.

          Show
          stack added a comment - This is in way of 0.92 release.
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-04 22:29:08, Andrew Purtell wrote:

          > security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java, line 66

          > <https://reviews.apache.org/r/2041/diff/3/?file=55459#file55459line66>

          >

          > Would 1 slave be sufficient?

          Yes, no reason for more than 1.

          On 2011-11-04 22:29:08, Andrew Purtell wrote:

          > security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java, line 87

          > <https://reviews.apache.org/r/2041/diff/3/?file=55460#file55460line87>

          >

          > Should we wait for the ACL table to become available here?

          >

          > I've seen this after making changes that alter connection setup timing:

          >

          > org.apache.hadoop.hbase.TableNotFoundException: acl

          > at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:863)

          > at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:732)

          > at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:697)

          > at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:196)

          > at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:152)

          > at org.apache.hadoop.hbase.security.rbac.TestAccessController.setupBeforeClass(TestAccessController.java:95)

          Ok, will make sure we wait until it's available.

          • Gary

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review3057
          -----------------------------------------------------------

          On 2011-11-01 21:18:27, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-01 21:18:27)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          src/main/resources/hbase-default.xml 3785533

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 61e04d8

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-04 22:29:08, Andrew Purtell wrote: > security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java, line 66 > < https://reviews.apache.org/r/2041/diff/3/?file=55459#file55459line66 > > > Would 1 slave be sufficient? Yes, no reason for more than 1. On 2011-11-04 22:29:08, Andrew Purtell wrote: > security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java, line 87 > < https://reviews.apache.org/r/2041/diff/3/?file=55460#file55460line87 > > > Should we wait for the ACL table to become available here? > > I've seen this after making changes that alter connection setup timing: > > org.apache.hadoop.hbase.TableNotFoundException: acl > at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:863) > at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:732) > at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:697) > at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:196) > at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:152) > at org.apache.hadoop.hbase.security.rbac.TestAccessController.setupBeforeClass(TestAccessController.java:95) Ok, will make sure we wait until it's available. Gary ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review3057 ----------------------------------------------------------- On 2011-11-01 21:18:27, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-01 21:18:27) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/resources/hbase-default.xml 3785533 src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 61e04d8 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review3057
          -----------------------------------------------------------

          Ship it!

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java
          <https://reviews.apache.org/r/2041/#comment6817>

          Would 1 slave be sufficient?

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
          <https://reviews.apache.org/r/2041/#comment6818>

          Should we wait for the ACL table to become available here?

          I've seen this after making changes that alter connection setup timing:

          org.apache.hadoop.hbase.TableNotFoundException: acl
          at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:863)
          at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:732)
          at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:697)
          at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:196)
          at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:152)
          at org.apache.hadoop.hbase.security.rbac.TestAccessController.setupBeforeClass(TestAccessController.java:95)

          • Andrew

          On 2011-11-01 21:18:27, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-01 21:18:27)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          src/main/resources/hbase-default.xml 3785533

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 61e04d8

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review3057 ----------------------------------------------------------- Ship it! security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java < https://reviews.apache.org/r/2041/#comment6817 > Would 1 slave be sufficient? security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java < https://reviews.apache.org/r/2041/#comment6818 > Should we wait for the ACL table to become available here? I've seen this after making changes that alter connection setup timing: org.apache.hadoop.hbase.TableNotFoundException: acl at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:863) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:732) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:697) at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:196) at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:152) at org.apache.hadoop.hbase.security.rbac.TestAccessController.setupBeforeClass(TestAccessController.java:95) Andrew On 2011-11-01 21:18:27, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-01 21:18:27) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/resources/hbase-default.xml 3785533 src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 61e04d8 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 192

          > <https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line192>

          >

          > Debug logging should go to LOG not AUDITLOG

          Gary Helmling wrote:

          The idea was that all authorization decisions should be separated into audit log. Here we're allowing access, so AUDITLOG seemed to make sense. I agree that this still needs to be cleaned up a lot. Maybe all audit logging should be done up in requirePermission() with authorization result? At the very least we need a consistent format and consistent logging levels for messages (trace, right?).

          Andrew Purtell wrote:

          > Maybe all audit logging should be done up in requirePermission() with authorization result?

          Sounds good.

          > At the very least we need a consistent format and consistent logging levels for messages (trace, right?).

          I'd argue for TRACE

          Gary Helmling wrote:

          Reworked the audit logging to happen in requirePermission(), so we get a single log message per auth check indicating success or failure, with a more consistent format. Result is logged to AUDITLOG at trace level.

          Michael Stack wrote:

          Is there TRACE level in our commons interface? I believe it just maps to DEBUG?

          Gary Helmling wrote:

          Commons-logging source for 1.1.1 claims that with log4j >= 1.2.12, trace level is supported. Prior to that it's mapped to debug.

          Oh. We need TRACE bad. We have 1.2.16 log4j. Have you seen TRACE logs Gary? If so, that'd make me happy.

          • Michael

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review2077
          -----------------------------------------------------------

          On 2011-11-01 21:18:27, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-01 21:18:27)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          src/main/resources/hbase-default.xml 3785533

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 61e04d8

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 192 > < https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line192 > > > Debug logging should go to LOG not AUDITLOG Gary Helmling wrote: The idea was that all authorization decisions should be separated into audit log. Here we're allowing access, so AUDITLOG seemed to make sense. I agree that this still needs to be cleaned up a lot. Maybe all audit logging should be done up in requirePermission() with authorization result? At the very least we need a consistent format and consistent logging levels for messages (trace, right?). Andrew Purtell wrote: > Maybe all audit logging should be done up in requirePermission() with authorization result? Sounds good. > At the very least we need a consistent format and consistent logging levels for messages (trace, right?). I'd argue for TRACE Gary Helmling wrote: Reworked the audit logging to happen in requirePermission(), so we get a single log message per auth check indicating success or failure, with a more consistent format. Result is logged to AUDITLOG at trace level. Michael Stack wrote: Is there TRACE level in our commons interface? I believe it just maps to DEBUG? Gary Helmling wrote: Commons-logging source for 1.1.1 claims that with log4j >= 1.2.12, trace level is supported. Prior to that it's mapped to debug. Oh. We need TRACE bad. We have 1.2.16 log4j. Have you seen TRACE logs Gary? If so, that'd make me happy. Michael ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review2077 ----------------------------------------------------------- On 2011-11-01 21:18:27, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-01 21:18:27) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/resources/hbase-default.xml 3785533 src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 61e04d8 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/
          -----------------------------------------------------------

          (Updated 2011-11-01 21:18:27.775440)

          Review request for hbase.

          Changes
          -------

          Fixed the security class package names (o.a.h.h.security.rbac -> o.a.h.h.security.access) in src/main/ruby/hbase/security.rb

          Summary
          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          • AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.
          • AccessController -
          • implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.
          • implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.
          • ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.
          • Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands
          • Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.
          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs (updated)


          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8
          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53
          src/main/resources/hbase-default.xml 3785533
          src/main/ruby/hbase.rb 4d27191
          src/main/ruby/hbase/admin.rb 61e04d8
          src/main/ruby/hbase/hbase.rb beb2450
          src/main/ruby/hbase/security.rb PRE-CREATION
          src/main/ruby/shell.rb 9a47600
          src/main/ruby/shell/commands.rb a352c2e
          src/main/ruby/shell/commands/grant.rb PRE-CREATION
          src/main/ruby/shell/commands/revoke.rb PRE-CREATION
          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing
          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-01 21:18:27.775440) Review request for hbase. Changes ------- Fixed the security class package names (o.a.h.h.security.rbac -> o.a.h.h.security.access) in src/main/ruby/hbase/security.rb Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. AccessController - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs (updated) security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/resources/hbase-default.xml 3785533 src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 61e04d8 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 192

          > <https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line192>

          >

          > Debug logging should go to LOG not AUDITLOG

          Gary Helmling wrote:

          The idea was that all authorization decisions should be separated into audit log. Here we're allowing access, so AUDITLOG seemed to make sense. I agree that this still needs to be cleaned up a lot. Maybe all audit logging should be done up in requirePermission() with authorization result? At the very least we need a consistent format and consistent logging levels for messages (trace, right?).

          Andrew Purtell wrote:

          > Maybe all audit logging should be done up in requirePermission() with authorization result?

          Sounds good.

          > At the very least we need a consistent format and consistent logging levels for messages (trace, right?).

          I'd argue for TRACE

          Gary Helmling wrote:

          Reworked the audit logging to happen in requirePermission(), so we get a single log message per auth check indicating success or failure, with a more consistent format. Result is logged to AUDITLOG at trace level.

          Michael Stack wrote:

          Is there TRACE level in our commons interface? I believe it just maps to DEBUG?

          Commons-logging source for 1.1.1 claims that with log4j >= 1.2.12, trace level is supported. Prior to that it's mapped to debug.

          • Gary

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review2077
          -----------------------------------------------------------

          On 2011-11-01 00:26:37, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-01 00:26:37)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          src/main/resources/hbase-default.xml 3785533

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 61e04d8

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 192 > < https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line192 > > > Debug logging should go to LOG not AUDITLOG Gary Helmling wrote: The idea was that all authorization decisions should be separated into audit log. Here we're allowing access, so AUDITLOG seemed to make sense. I agree that this still needs to be cleaned up a lot. Maybe all audit logging should be done up in requirePermission() with authorization result? At the very least we need a consistent format and consistent logging levels for messages (trace, right?). Andrew Purtell wrote: > Maybe all audit logging should be done up in requirePermission() with authorization result? Sounds good. > At the very least we need a consistent format and consistent logging levels for messages (trace, right?). I'd argue for TRACE Gary Helmling wrote: Reworked the audit logging to happen in requirePermission(), so we get a single log message per auth check indicating success or failure, with a more consistent format. Result is logged to AUDITLOG at trace level. Michael Stack wrote: Is there TRACE level in our commons interface? I believe it just maps to DEBUG? Commons-logging source for 1.1.1 claims that with log4j >= 1.2.12, trace level is supported. Prior to that it's mapped to debug. Gary ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review2077 ----------------------------------------------------------- On 2011-11-01 00:26:37, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-01 00:26:37) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/resources/hbase-default.xml 3785533 src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 61e04d8 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 192

          > <https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line192>

          >

          > Debug logging should go to LOG not AUDITLOG

          Gary Helmling wrote:

          The idea was that all authorization decisions should be separated into audit log. Here we're allowing access, so AUDITLOG seemed to make sense. I agree that this still needs to be cleaned up a lot. Maybe all audit logging should be done up in requirePermission() with authorization result? At the very least we need a consistent format and consistent logging levels for messages (trace, right?).

          Andrew Purtell wrote:

          > Maybe all audit logging should be done up in requirePermission() with authorization result?

          Sounds good.

          > At the very least we need a consistent format and consistent logging levels for messages (trace, right?).

          I'd argue for TRACE

          Gary Helmling wrote:

          Reworked the audit logging to happen in requirePermission(), so we get a single log message per auth check indicating success or failure, with a more consistent format. Result is logged to AUDITLOG at trace level.

          Is there TRACE level in our commons interface? I believe it just maps to DEBUG?

          • Michael

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review2077
          -----------------------------------------------------------

          On 2011-11-01 00:26:37, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-11-01 00:26:37)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          src/main/resources/hbase-default.xml 3785533

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb 61e04d8

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 192 > < https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line192 > > > Debug logging should go to LOG not AUDITLOG Gary Helmling wrote: The idea was that all authorization decisions should be separated into audit log. Here we're allowing access, so AUDITLOG seemed to make sense. I agree that this still needs to be cleaned up a lot. Maybe all audit logging should be done up in requirePermission() with authorization result? At the very least we need a consistent format and consistent logging levels for messages (trace, right?). Andrew Purtell wrote: > Maybe all audit logging should be done up in requirePermission() with authorization result? Sounds good. > At the very least we need a consistent format and consistent logging levels for messages (trace, right?). I'd argue for TRACE Gary Helmling wrote: Reworked the audit logging to happen in requirePermission(), so we get a single log message per auth check indicating success or failure, with a more consistent format. Result is logged to AUDITLOG at trace level. Is there TRACE level in our commons interface? I believe it just maps to DEBUG? Michael ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review2077 ----------------------------------------------------------- On 2011-11-01 00:26:37, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-01 00:26:37) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/resources/hbase-default.xml 3785533 src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 61e04d8 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/
          -----------------------------------------------------------

          (Updated 2011-11-01 00:26:37.040678)

          Review request for hbase.

          Changes
          -------

          Updated patch addressing review comments:

          • cleaned up audit logging of authorization decisions. Logging now occurs in AccessController.requirePermission(), with a single audit log entry per authorization decision.
          • audit logging uses a more consistent format
          • KeeperExceptions in ZKPermissionWatcher now trigger aborts where necessary, instead of logging and dropping the exceptions.

          Summary
          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          • AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.
          • AccessController -
          • implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.
          • implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.
          • ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.
          • Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands
          • Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.
          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs (updated)


          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8
          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53
          src/main/resources/hbase-default.xml 3785533
          src/main/ruby/hbase.rb 4d27191
          src/main/ruby/hbase/admin.rb 61e04d8
          src/main/ruby/hbase/hbase.rb beb2450
          src/main/ruby/hbase/security.rb PRE-CREATION
          src/main/ruby/shell.rb 9a47600
          src/main/ruby/shell/commands.rb a352c2e
          src/main/ruby/shell/commands/grant.rb PRE-CREATION
          src/main/ruby/shell/commands/revoke.rb PRE-CREATION
          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          Diff: https://reviews.apache.org/r/2041/diff

          Testing
          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-11-01 00:26:37.040678) Review request for hbase. Changes ------- Updated patch addressing review comments: cleaned up audit logging of authorization decisions. Logging now occurs in AccessController.requirePermission(), with a single audit log entry per authorization decision. audit logging uses a more consistent format KeeperExceptions in ZKPermissionWatcher now trigger aborts where necessary, instead of logging and dropping the exceptions. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. AccessController - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs (updated) security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/access/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/access/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java 8a40762 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/resources/hbase-default.xml 3785533 src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb 61e04d8 src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java, line 84

          > <https://reviews.apache.org/r/2041/diff/1/?file=45407#file45407line84>

          >

          > Isn't this an error?

          Gary Helmling wrote:

          Yes, and in this context a pretty bad one, as it probably means region server initiated RPCs won't work or will be denied. We should probably let the IOE escape here...

          Andrew Purtell wrote:

          Agree.

          Letting the IOException from getCurrent() escape and throwing an exception if the returned user is null.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java, line 113

          > <https://reviews.apache.org/r/2041/diff/1/?file=45407#file45407line113>

          >

          > Should be at DEBUG level

          Done.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlFilter.java, line 37

          > <https://reviews.apache.org/r/2041/diff/1/?file=45402#file45402line37>

          >

          > Could be stated better.

          Fixed.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlLists.java, line 98

          > <https://reviews.apache.org/r/2041/diff/1/?file=45403#file45403line98>

          >

          > Comment needs updating.

          >

          Fixed.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 98

          > <https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line98>

          >

          > Can we make this 1?

          Gary Helmling wrote:

          sure

          Done.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 192

          > <https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line192>

          >

          > Debug logging should go to LOG not AUDITLOG

          Gary Helmling wrote:

          The idea was that all authorization decisions should be separated into audit log. Here we're allowing access, so AUDITLOG seemed to make sense. I agree that this still needs to be cleaned up a lot. Maybe all audit logging should be done up in requirePermission() with authorization result? At the very least we need a consistent format and consistent logging levels for messages (trace, right?).

          Andrew Purtell wrote:

          > Maybe all audit logging should be done up in requirePermission() with authorization result?

          Sounds good.

          > At the very least we need a consistent format and consistent logging levels for messages (trace, right?).

          I'd argue for TRACE

          Reworked the audit logging to happen in requirePermission(), so we get a single log message per auth check indicating success or failure, with a more consistent format. Result is logged to AUDITLOG at trace level.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 366

          > <https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line366>

          >

          > Should hasFamilyQualifierPermission log to AUDITLOG? It is used in places to make decisions – an exception is thrown directly or not.

          Gary Helmling wrote:

          Yes, agree, we should either log to AUDITLOG at decision points here or consistently move the AUDITLOG logging up a level out of permissionGranted() and hasFamilyQualifierPermission().

          With moving the audit logging up to requirePermission(), logging to AUDITLOG here would be redundant. Removing the existing log message.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 375

          > <https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line375>

          >

          > Another one of these was sent to AUDITLOG above. Do the same here? Should be INFO or TRACE level? TRACE makes more sense to me.

          Gary Helmling wrote:

          Agree, should go to AUDITLOG at trace.

          Removing as redundant with the checking in permissionGranted() and AUDITLOG logging performed in requirePermission().

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 497

          > <https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line497>

          >

          > Ultimately users should be allowed to enable or disable their own tables, but only after such operations don't carry as much systemic risk as they do currently.

          >

          > In that case, CREATE permission and an ownership check could follow the test for ADMIN permission.

          Noted

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 590

          > <https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line590>

          >

          > Should be logged with ERROR?

          Gary Helmling wrote:

          sure

          Done.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 832

          > <https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line832>

          >

          > Should this go to AUDITLOG? At INFO or TRACE level? My preference is TRACE.

          Done.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 856

          > <https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line856>

          >

          > Should this go to AUDITLOG? At INFO or TRACE level? My preference is TRACE.

          Gary Helmling wrote:

          Yes, agree.

          Done.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java, line 77

          > <https://reviews.apache.org/r/2041/diff/1/?file=45410#file45410line77>

          >

          > Shouldn't we propagate ZK exceptions upward? or at least convert them to IOE and throw those? Otherwise the permission cache is silently at risk of being out of sync with the ACL table.

          >

          > The safest thing to do is force a region close by bubbling up an exception from the coprocessor. This assumes that the coprocessor framework or regionserver will trigger a region close if it receives an unhandled exception from coprocessor code, and that this won't down the whole regionserver.

          Gary Helmling wrote:

          Yes, shouldn't just be swallowing this.

          ZooKeeperListener defined methods don't throw any exceptions, so logging and aborting on KeeperExceptions here instead.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java, line 59

          > <https://reviews.apache.org/r/2041/diff/1/?file=45410#file45410line59>

          >

          > I wonder if there is some way we can check if a secure variant of ZooKeeper is running, and refuse to initialize if not.

          Gary Helmling wrote:

          My thinking has been to handle all secure ZooKeeper changes separately. So I'd prefer to handle any check here as part of that.

          I do think it's reasonable to run AccessController with only SIMPLE auth and no secure ZooKeeper. It's not secure but could still be useful (we currently use this setup for tests).

          We could complain loudly to give an indication that you have a security hole though.

          Andrew Purtell wrote:

          > I do think it's reasonable to run AccessController with only SIMPLE auth and no secure ZooKeeper.

          I'd argue only for test cases, and we can make provisions for tests to add an undocumented configuration property to that effect.

          > We could complain loudly to give an indication that you have a security hole though.

          Complaining loudly is good to do in any case except when unit tests want to do something.

          I will add this into a separate patch handling ZooKeeper authentication.

          • Gary

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review2077
          -----------------------------------------------------------

          On 2011-09-23 19:14:20, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-09-23 19:14:20)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 46a1a3d

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 699a5f5

          src/main/resources/hbase-default.xml 2c8f44b

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb b244ffe

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/table_permission.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java 4d7ee22

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java, line 84 > < https://reviews.apache.org/r/2041/diff/1/?file=45407#file45407line84 > > > Isn't this an error? Gary Helmling wrote: Yes, and in this context a pretty bad one, as it probably means region server initiated RPCs won't work or will be denied. We should probably let the IOE escape here... Andrew Purtell wrote: Agree. Letting the IOException from getCurrent() escape and throwing an exception if the returned user is null. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java, line 113 > < https://reviews.apache.org/r/2041/diff/1/?file=45407#file45407line113 > > > Should be at DEBUG level Done. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlFilter.java, line 37 > < https://reviews.apache.org/r/2041/diff/1/?file=45402#file45402line37 > > > Could be stated better. Fixed. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlLists.java, line 98 > < https://reviews.apache.org/r/2041/diff/1/?file=45403#file45403line98 > > > Comment needs updating. > Fixed. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 98 > < https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line98 > > > Can we make this 1? Gary Helmling wrote: sure Done. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 192 > < https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line192 > > > Debug logging should go to LOG not AUDITLOG Gary Helmling wrote: The idea was that all authorization decisions should be separated into audit log. Here we're allowing access, so AUDITLOG seemed to make sense. I agree that this still needs to be cleaned up a lot. Maybe all audit logging should be done up in requirePermission() with authorization result? At the very least we need a consistent format and consistent logging levels for messages (trace, right?). Andrew Purtell wrote: > Maybe all audit logging should be done up in requirePermission() with authorization result? Sounds good. > At the very least we need a consistent format and consistent logging levels for messages (trace, right?). I'd argue for TRACE Reworked the audit logging to happen in requirePermission(), so we get a single log message per auth check indicating success or failure, with a more consistent format. Result is logged to AUDITLOG at trace level. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 366 > < https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line366 > > > Should hasFamilyQualifierPermission log to AUDITLOG? It is used in places to make decisions – an exception is thrown directly or not. Gary Helmling wrote: Yes, agree, we should either log to AUDITLOG at decision points here or consistently move the AUDITLOG logging up a level out of permissionGranted() and hasFamilyQualifierPermission(). With moving the audit logging up to requirePermission(), logging to AUDITLOG here would be redundant. Removing the existing log message. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 375 > < https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line375 > > > Another one of these was sent to AUDITLOG above. Do the same here? Should be INFO or TRACE level? TRACE makes more sense to me. Gary Helmling wrote: Agree, should go to AUDITLOG at trace. Removing as redundant with the checking in permissionGranted() and AUDITLOG logging performed in requirePermission(). On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 497 > < https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line497 > > > Ultimately users should be allowed to enable or disable their own tables, but only after such operations don't carry as much systemic risk as they do currently. > > In that case, CREATE permission and an ownership check could follow the test for ADMIN permission. Noted On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 590 > < https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line590 > > > Should be logged with ERROR? Gary Helmling wrote: sure Done. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 832 > < https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line832 > > > Should this go to AUDITLOG? At INFO or TRACE level? My preference is TRACE. Done. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 856 > < https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line856 > > > Should this go to AUDITLOG? At INFO or TRACE level? My preference is TRACE. Gary Helmling wrote: Yes, agree. Done. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java, line 77 > < https://reviews.apache.org/r/2041/diff/1/?file=45410#file45410line77 > > > Shouldn't we propagate ZK exceptions upward? or at least convert them to IOE and throw those? Otherwise the permission cache is silently at risk of being out of sync with the ACL table. > > The safest thing to do is force a region close by bubbling up an exception from the coprocessor. This assumes that the coprocessor framework or regionserver will trigger a region close if it receives an unhandled exception from coprocessor code, and that this won't down the whole regionserver. Gary Helmling wrote: Yes, shouldn't just be swallowing this. ZooKeeperListener defined methods don't throw any exceptions, so logging and aborting on KeeperExceptions here instead. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java, line 59 > < https://reviews.apache.org/r/2041/diff/1/?file=45410#file45410line59 > > > I wonder if there is some way we can check if a secure variant of ZooKeeper is running, and refuse to initialize if not. Gary Helmling wrote: My thinking has been to handle all secure ZooKeeper changes separately. So I'd prefer to handle any check here as part of that. I do think it's reasonable to run AccessController with only SIMPLE auth and no secure ZooKeeper. It's not secure but could still be useful (we currently use this setup for tests). We could complain loudly to give an indication that you have a security hole though. Andrew Purtell wrote: > I do think it's reasonable to run AccessController with only SIMPLE auth and no secure ZooKeeper. I'd argue only for test cases, and we can make provisions for tests to add an undocumented configuration property to that effect. > We could complain loudly to give an indication that you have a security hole though. Complaining loudly is good to do in any case except when unit tests want to do something. I will add this into a separate patch handling ZooKeeper authentication. Gary ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review2077 ----------------------------------------------------------- On 2011-09-23 19:14:20, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-09-23 19:14:20) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 46a1a3d src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 699a5f5 src/main/resources/hbase-default.xml 2c8f44b src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb b244ffe src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/table_permission.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java 4d7ee22 Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 192

          > <https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line192>

          >

          > Debug logging should go to LOG not AUDITLOG

          Gary Helmling wrote:

          The idea was that all authorization decisions should be separated into audit log. Here we're allowing access, so AUDITLOG seemed to make sense. I agree that this still needs to be cleaned up a lot. Maybe all audit logging should be done up in requirePermission() with authorization result? At the very least we need a consistent format and consistent logging levels for messages (trace, right?).

          Maybe all audit logging should be done up in requirePermission() with authorization result?

          Sounds good.

          At the very least we need a consistent format and consistent logging levels for messages (trace, right?).

          I'd argue for TRACE

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java, line 47

          > <https://reviews.apache.org/r/2041/diff/1/?file=45407#file45407line47>

          >

          > Maybe we can call this ".auth."? We don't really have an RBAC implementation yet. Likewise for the package name for all of this stuff? Just a random thought.

          Gary Helmling wrote:

          Yeah "rbac" here and in package name is a misnomer. How about using "access" instead? "auth" seems ambiguous to me as it could mean "authentication" or "authorization". JDK uses "auth" in javax.security.auth and claims it's for both, but seems like that and sub-packages are more "authentication" related to me. Hadoop uses "authorize" for a similar package to this.

          "access" sounds good to me.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java, line 84

          > <https://reviews.apache.org/r/2041/diff/1/?file=45407#file45407line84>

          >

          > Isn't this an error?

          Gary Helmling wrote:

          Yes, and in this context a pretty bad one, as it probably means region server initiated RPCs won't work or will be denied. We should probably let the IOE escape here...

          Agree.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/Permission.java, line 174

          > <https://reviews.apache.org/r/2041/diff/1/?file=45406#file45406line174>

          >

          > What if instead we check for version 0 and throw an IllegalArgumentException if so? Technically, it is an invalid request if it contains an unrecognizable action code. Skipping this check if version > 0 would be a way to handle new perms while not accepting incorrect input otherwise.

          Gary Helmling wrote:

          Yeah, seems safer to throw an exception here than to ignore invalid input. What about throwing an IOException (to tie in to existing error handling)?

          We could potentially trap the VersionMismatchException from VersionedWritable to allow skip and continue when reading newer versions of Permission with potentially added Action codes. Would need to think about what kind of errors that would expose us to.

          What about throwing an IOException (to tie in to existing error handling)?

          Throwing an IOE sounds good.

          We could potentially trap the VersionMismatchException from VersionedWritable to allow skip and continue when reading newer versions of Permission with potentially added Action codes.

          I think that is reasonable, with something logged at WARN level. The idea here is to ride over a rolling restart. Would not see long term operation with mismatching versions.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java, line 59

          > <https://reviews.apache.org/r/2041/diff/1/?file=45410#file45410line59>

          >

          > I wonder if there is some way we can check if a secure variant of ZooKeeper is running, and refuse to initialize if not.

          Gary Helmling wrote:

          My thinking has been to handle all secure ZooKeeper changes separately. So I'd prefer to handle any check here as part of that.

          I do think it's reasonable to run AccessController with only SIMPLE auth and no secure ZooKeeper. It's not secure but could still be useful (we currently use this setup for tests).

          We could complain loudly to give an indication that you have a security hole though.

          I do think it's reasonable to run AccessController with only SIMPLE auth and no secure ZooKeeper.

          I'd argue only for test cases, and we can make provisions for tests to add an undocumented configuration property to that effect.

          We could complain loudly to give an indication that you have a security hole though.

          Complaining loudly is good to do in any case except when unit tests want to do something.

          • Andrew

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review2077
          -----------------------------------------------------------

          On 2011-09-23 19:14:20, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-09-23 19:14:20)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 46a1a3d

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 699a5f5

          src/main/resources/hbase-default.xml 2c8f44b

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb b244ffe

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/table_permission.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java 4d7ee22

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 192 > < https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line192 > > > Debug logging should go to LOG not AUDITLOG Gary Helmling wrote: The idea was that all authorization decisions should be separated into audit log. Here we're allowing access, so AUDITLOG seemed to make sense. I agree that this still needs to be cleaned up a lot. Maybe all audit logging should be done up in requirePermission() with authorization result? At the very least we need a consistent format and consistent logging levels for messages (trace, right?). Maybe all audit logging should be done up in requirePermission() with authorization result? Sounds good. At the very least we need a consistent format and consistent logging levels for messages (trace, right?). I'd argue for TRACE On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java, line 47 > < https://reviews.apache.org/r/2041/diff/1/?file=45407#file45407line47 > > > Maybe we can call this ".auth."? We don't really have an RBAC implementation yet. Likewise for the package name for all of this stuff? Just a random thought. Gary Helmling wrote: Yeah "rbac" here and in package name is a misnomer. How about using "access" instead? "auth" seems ambiguous to me as it could mean "authentication" or "authorization". JDK uses "auth" in javax.security.auth and claims it's for both, but seems like that and sub-packages are more "authentication" related to me. Hadoop uses "authorize" for a similar package to this. "access" sounds good to me. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java, line 84 > < https://reviews.apache.org/r/2041/diff/1/?file=45407#file45407line84 > > > Isn't this an error? Gary Helmling wrote: Yes, and in this context a pretty bad one, as it probably means region server initiated RPCs won't work or will be denied. We should probably let the IOE escape here... Agree. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/Permission.java, line 174 > < https://reviews.apache.org/r/2041/diff/1/?file=45406#file45406line174 > > > What if instead we check for version 0 and throw an IllegalArgumentException if so? Technically, it is an invalid request if it contains an unrecognizable action code. Skipping this check if version > 0 would be a way to handle new perms while not accepting incorrect input otherwise. Gary Helmling wrote: Yeah, seems safer to throw an exception here than to ignore invalid input. What about throwing an IOException (to tie in to existing error handling)? We could potentially trap the VersionMismatchException from VersionedWritable to allow skip and continue when reading newer versions of Permission with potentially added Action codes. Would need to think about what kind of errors that would expose us to. What about throwing an IOException (to tie in to existing error handling)? Throwing an IOE sounds good. We could potentially trap the VersionMismatchException from VersionedWritable to allow skip and continue when reading newer versions of Permission with potentially added Action codes. I think that is reasonable, with something logged at WARN level. The idea here is to ride over a rolling restart. Would not see long term operation with mismatching versions. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java, line 59 > < https://reviews.apache.org/r/2041/diff/1/?file=45410#file45410line59 > > > I wonder if there is some way we can check if a secure variant of ZooKeeper is running, and refuse to initialize if not. Gary Helmling wrote: My thinking has been to handle all secure ZooKeeper changes separately. So I'd prefer to handle any check here as part of that. I do think it's reasonable to run AccessController with only SIMPLE auth and no secure ZooKeeper. It's not secure but could still be useful (we currently use this setup for tests). We could complain loudly to give an indication that you have a security hole though. I do think it's reasonable to run AccessController with only SIMPLE auth and no secure ZooKeeper. I'd argue only for test cases, and we can make provisions for tests to add an undocumented configuration property to that effect. We could complain loudly to give an indication that you have a security hole though. Complaining loudly is good to do in any case except when unit tests want to do something. Andrew ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review2077 ----------------------------------------------------------- On 2011-09-23 19:14:20, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-09-23 19:14:20) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 46a1a3d src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 699a5f5 src/main/resources/hbase-default.xml 2c8f44b src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb b244ffe src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/table_permission.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java 4d7ee22 Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > Looks good. The majority of my comments have to do with inconsistent logging practice.

          Thanks for the review. I'll post an update with some cleanups and some reworking of the AUDITLOG handling.

          • Gary

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review2077
          -----------------------------------------------------------

          On 2011-09-23 19:14:20, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-09-23 19:14:20)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 46a1a3d

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 699a5f5

          src/main/resources/hbase-default.xml 2c8f44b

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb b244ffe

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/table_permission.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java 4d7ee22

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-09-27 16:58:47, Andrew Purtell wrote: > Looks good. The majority of my comments have to do with inconsistent logging practice. Thanks for the review. I'll post an update with some cleanups and some reworking of the AUDITLOG handling. Gary ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review2077 ----------------------------------------------------------- On 2011-09-23 19:14:20, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-09-23 19:14:20) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 46a1a3d src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 699a5f5 src/main/resources/hbase-default.xml 2c8f44b src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb b244ffe src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/table_permission.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java 4d7ee22 Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 98

          > <https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line98>

          >

          > Can we make this 1?

          sure

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 192

          > <https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line192>

          >

          > Debug logging should go to LOG not AUDITLOG

          The idea was that all authorization decisions should be separated into audit log. Here we're allowing access, so AUDITLOG seemed to make sense. I agree that this still needs to be cleaned up a lot. Maybe all audit logging should be done up in requirePermission() with authorization result? At the very least we need a consistent format and consistent logging levels for messages (trace, right?).

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 200

          > <https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line200>

          >

          > Should be INFO or TRACE level? TRACE makes more sense to me.

          Sure, can use trace for all audit log decisions.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 208

          > <https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line208>

          >

          > Debug logging should go to LOG not AUDITLOG

          This is an authorization decision since we're returning true below. We can make this trace level, and improve the format, but I think AUDITLOG (if enabled) should contain a single message per request on why the request was allowed or denied.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 274

          > <https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line274>

          >

          > Should be INFO or TRACE level? TRACE makes more sense to me.

          will change to trace.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 354

          > <https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line354>

          >

          > Should something go to AUDITLOG here?

          Failure should already have been recorded in AUDITLOG via logDenied(). Agree that moving AUDITLOG messages up here with consistent format would be clearer, but will require some restructuring of return value from permissionGranted() so that some context specific reason can be pulled back up for logging.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 366

          > <https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line366>

          >

          > Should hasFamilyQualifierPermission log to AUDITLOG? It is used in places to make decisions – an exception is thrown directly or not.

          Yes, agree, we should either log to AUDITLOG at decision points here or consistently move the AUDITLOG logging up a level out of permissionGranted() and hasFamilyQualifierPermission().

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 375

          > <https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line375>

          >

          > Another one of these was sent to AUDITLOG above. Do the same here? Should be INFO or TRACE level? TRACE makes more sense to me.

          Agree, should go to AUDITLOG at trace.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 590

          > <https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line590>

          >

          > Should be logged with ERROR?

          sure

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 856

          > <https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line856>

          >

          > Should this go to AUDITLOG? At INFO or TRACE level? My preference is TRACE.

          Yes, agree.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/Permission.java, line 174

          > <https://reviews.apache.org/r/2041/diff/1/?file=45406#file45406line174>

          >

          > What if instead we check for version 0 and throw an IllegalArgumentException if so? Technically, it is an invalid request if it contains an unrecognizable action code. Skipping this check if version > 0 would be a way to handle new perms while not accepting incorrect input otherwise.

          Yeah, seems safer to throw an exception here than to ignore invalid input. What about throwing an IOException (to tie in to existing error handling)?

          We could potentially trap the VersionMismatchException from VersionedWritable to allow skip and continue when reading newer versions of Permission with potentially added Action codes. Would need to think about what kind of errors that would expose us to.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java, line 47

          > <https://reviews.apache.org/r/2041/diff/1/?file=45407#file45407line47>

          >

          > Maybe we can call this ".auth."? We don't really have an RBAC implementation yet. Likewise for the package name for all of this stuff? Just a random thought.

          Yeah "rbac" here and in package name is a misnomer. How about using "access" instead? "auth" seems ambiguous to me as it could mean "authentication" or "authorization". JDK uses "auth" in javax.security.auth and claims it's for both, but seems like that and sub-packages are more "authentication" related to me. Hadoop uses "authorize" for a similar package to this.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java, line 84

          > <https://reviews.apache.org/r/2041/diff/1/?file=45407#file45407line84>

          >

          > Isn't this an error?

          Yes, and in this context a pretty bad one, as it probably means region server initiated RPCs won't work or will be denied. We should probably let the IOE escape here...

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java, line 59

          > <https://reviews.apache.org/r/2041/diff/1/?file=45410#file45410line59>

          >

          > I wonder if there is some way we can check if a secure variant of ZooKeeper is running, and refuse to initialize if not.

          My thinking has been to handle all secure ZooKeeper changes separately. So I'd prefer to handle any check here as part of that.

          I do think it's reasonable to run AccessController with only SIMPLE auth and no secure ZooKeeper. It's not secure but could still be useful (we currently use this setup for tests).

          We could complain loudly to give an indication that you have a security hole though.

          On 2011-09-27 16:58:47, Andrew Purtell wrote:

          > security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java, line 77

          > <https://reviews.apache.org/r/2041/diff/1/?file=45410#file45410line77>

          >

          > Shouldn't we propagate ZK exceptions upward? or at least convert them to IOE and throw those? Otherwise the permission cache is silently at risk of being out of sync with the ACL table.

          >

          > The safest thing to do is force a region close by bubbling up an exception from the coprocessor. This assumes that the coprocessor framework or regionserver will trigger a region close if it receives an unhandled exception from coprocessor code, and that this won't down the whole regionserver.

          Yes, shouldn't just be swallowing this.

          • Gary

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review2077
          -----------------------------------------------------------

          On 2011-09-23 19:14:20, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-09-23 19:14:20)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 46a1a3d

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 699a5f5

          src/main/resources/hbase-default.xml 2c8f44b

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb b244ffe

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/table_permission.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java 4d7ee22

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 98 > < https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line98 > > > Can we make this 1? sure On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 192 > < https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line192 > > > Debug logging should go to LOG not AUDITLOG The idea was that all authorization decisions should be separated into audit log. Here we're allowing access, so AUDITLOG seemed to make sense. I agree that this still needs to be cleaned up a lot. Maybe all audit logging should be done up in requirePermission() with authorization result? At the very least we need a consistent format and consistent logging levels for messages (trace, right?). On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 200 > < https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line200 > > > Should be INFO or TRACE level? TRACE makes more sense to me. Sure, can use trace for all audit log decisions. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 208 > < https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line208 > > > Debug logging should go to LOG not AUDITLOG This is an authorization decision since we're returning true below. We can make this trace level, and improve the format, but I think AUDITLOG (if enabled) should contain a single message per request on why the request was allowed or denied. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 274 > < https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line274 > > > Should be INFO or TRACE level? TRACE makes more sense to me. will change to trace. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 354 > < https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line354 > > > Should something go to AUDITLOG here? Failure should already have been recorded in AUDITLOG via logDenied(). Agree that moving AUDITLOG messages up here with consistent format would be clearer, but will require some restructuring of return value from permissionGranted() so that some context specific reason can be pulled back up for logging. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 366 > < https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line366 > > > Should hasFamilyQualifierPermission log to AUDITLOG? It is used in places to make decisions – an exception is thrown directly or not. Yes, agree, we should either log to AUDITLOG at decision points here or consistently move the AUDITLOG logging up a level out of permissionGranted() and hasFamilyQualifierPermission(). On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 375 > < https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line375 > > > Another one of these was sent to AUDITLOG above. Do the same here? Should be INFO or TRACE level? TRACE makes more sense to me. Agree, should go to AUDITLOG at trace. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 590 > < https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line590 > > > Should be logged with ERROR? sure On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java, line 856 > < https://reviews.apache.org/r/2041/diff/1/?file=45404#file45404line856 > > > Should this go to AUDITLOG? At INFO or TRACE level? My preference is TRACE. Yes, agree. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/Permission.java, line 174 > < https://reviews.apache.org/r/2041/diff/1/?file=45406#file45406line174 > > > What if instead we check for version 0 and throw an IllegalArgumentException if so? Technically, it is an invalid request if it contains an unrecognizable action code. Skipping this check if version > 0 would be a way to handle new perms while not accepting incorrect input otherwise. Yeah, seems safer to throw an exception here than to ignore invalid input. What about throwing an IOException (to tie in to existing error handling)? We could potentially trap the VersionMismatchException from VersionedWritable to allow skip and continue when reading newer versions of Permission with potentially added Action codes. Would need to think about what kind of errors that would expose us to. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java, line 47 > < https://reviews.apache.org/r/2041/diff/1/?file=45407#file45407line47 > > > Maybe we can call this ".auth."? We don't really have an RBAC implementation yet. Likewise for the package name for all of this stuff? Just a random thought. Yeah "rbac" here and in package name is a misnomer. How about using "access" instead? "auth" seems ambiguous to me as it could mean "authentication" or "authorization". JDK uses "auth" in javax.security.auth and claims it's for both, but seems like that and sub-packages are more "authentication" related to me. Hadoop uses "authorize" for a similar package to this. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java, line 84 > < https://reviews.apache.org/r/2041/diff/1/?file=45407#file45407line84 > > > Isn't this an error? Yes, and in this context a pretty bad one, as it probably means region server initiated RPCs won't work or will be denied. We should probably let the IOE escape here... On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java, line 59 > < https://reviews.apache.org/r/2041/diff/1/?file=45410#file45410line59 > > > I wonder if there is some way we can check if a secure variant of ZooKeeper is running, and refuse to initialize if not. My thinking has been to handle all secure ZooKeeper changes separately. So I'd prefer to handle any check here as part of that. I do think it's reasonable to run AccessController with only SIMPLE auth and no secure ZooKeeper. It's not secure but could still be useful (we currently use this setup for tests). We could complain loudly to give an indication that you have a security hole though. On 2011-09-27 16:58:47, Andrew Purtell wrote: > security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java, line 77 > < https://reviews.apache.org/r/2041/diff/1/?file=45410#file45410line77 > > > Shouldn't we propagate ZK exceptions upward? or at least convert them to IOE and throw those? Otherwise the permission cache is silently at risk of being out of sync with the ACL table. > > The safest thing to do is force a region close by bubbling up an exception from the coprocessor. This assumes that the coprocessor framework or regionserver will trigger a region close if it receives an unhandled exception from coprocessor code, and that this won't down the whole regionserver. Yes, shouldn't just be swallowing this. Gary ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review2077 ----------------------------------------------------------- On 2011-09-23 19:14:20, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-09-23 19:14:20) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 46a1a3d src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 699a5f5 src/main/resources/hbase-default.xml 2c8f44b src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb b244ffe src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/table_permission.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java 4d7ee22 Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/#review2077
          -----------------------------------------------------------

          Looks good. The majority of my comments have to do with inconsistent logging practice.

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlFilter.java
          <https://reviews.apache.org/r/2041/#comment4718>

          Could be stated better.

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlLists.java
          <https://reviews.apache.org/r/2041/#comment4719>

          No.

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlLists.java
          <https://reviews.apache.org/r/2041/#comment4720>

          Comment needs updating.

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java
          <https://reviews.apache.org/r/2041/#comment4721>

          Can we make this 1?

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java
          <https://reviews.apache.org/r/2041/#comment4775>

          Debug logging should go to LOG not AUDITLOG

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java
          <https://reviews.apache.org/r/2041/#comment4782>

          Should be INFO or TRACE level? TRACE makes more sense to me.

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java
          <https://reviews.apache.org/r/2041/#comment4776>

          Debug logging should go to LOG not AUDITLOG

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java
          <https://reviews.apache.org/r/2041/#comment4779>

          Should be INFO or TRACE level? TRACE makes more sense to me.

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java
          <https://reviews.apache.org/r/2041/#comment4795>

          Should something go to AUDITLOG here?

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java
          <https://reviews.apache.org/r/2041/#comment4797>

          Should hasFamilyQualifierPermission log to AUDITLOG? It is used in places to make decisions – an exception is thrown directly or not.

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java
          <https://reviews.apache.org/r/2041/#comment4783>

          Another one of these was sent to AUDITLOG above. Do the same here? Should be INFO or TRACE level? TRACE makes more sense to me.

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java
          <https://reviews.apache.org/r/2041/#comment4786>

          Ultimately users should be allowed to enable or disable their own tables, but only after such operations don't carry as much systemic risk as they do currently.

          In that case, CREATE permission and an ownership check could follow the test for ADMIN permission.

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java
          <https://reviews.apache.org/r/2041/#comment4787>

          As above

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java
          <https://reviews.apache.org/r/2041/#comment4791>

          Should be logged with ERROR?

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java
          <https://reviews.apache.org/r/2041/#comment4799>

          Would it be clearer then to call permissionGranted() something like hasColumnsPermission() ? Just a random thought.

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java
          <https://reviews.apache.org/r/2041/#comment4803>

          Should this go to AUDITLOG? At INFO or TRACE level? My preference is TRACE.

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java
          <https://reviews.apache.org/r/2041/#comment4804>

          Should this go to AUDITLOG? At INFO or TRACE level? My preference is TRACE.

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/Permission.java
          <https://reviews.apache.org/r/2041/#comment4807>

          What if instead we check for version 0 and throw an IllegalArgumentException if so? Technically, it is an invalid request if it contains an unrecognizable action code. Skipping this check if version > 0 would be a way to handle new perms while not accepting incorrect input otherwise.

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java
          <https://reviews.apache.org/r/2041/#comment4813>

          Maybe we can call this ".auth."? We don't really have an RBAC implementation yet. Likewise for the package name for all of this stuff? Just a random thought.

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java
          <https://reviews.apache.org/r/2041/#comment4815>

          Isn't this an error?

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java
          <https://reviews.apache.org/r/2041/#comment4816>

          Should be at DEBUG level

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java
          <https://reviews.apache.org/r/2041/#comment4814>

          I wonder if there is some way we can check if a secure variant of ZooKeeper is running, and refuse to initialize if not.

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java
          <https://reviews.apache.org/r/2041/#comment4823>

          Shouldn't we propagate ZK exceptions upward? or at least convert them to IOE and throw those? Otherwise the permission cache is silently at risk of being out of sync with the ACL table.

          The safest thing to do is force a region close by bubbling up an exception from the coprocessor. This assumes that the coprocessor framework or regionserver will trigger a region close if it receives an unhandled exception from coprocessor code, and that this won't down the whole regionserver.

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java
          <https://reviews.apache.org/r/2041/#comment4825>

          As above

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java
          <https://reviews.apache.org/r/2041/#comment4826>

          As above

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java
          <https://reviews.apache.org/r/2041/#comment4827>

          As above

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
          <https://reviews.apache.org/r/2041/#comment4829>

          Yes I agree this makes sense here, for convenience in setting ownership through the existing alter functionality.

          • Andrew

          On 2011-09-23 19:14:20, Gary Helmling wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2041/

          -----------------------------------------------------------

          (Updated 2011-09-23 19:14:20)

          Review request for hbase.

          Summary

          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          * AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.

          * AccessController -

          - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.

          - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.

          * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.

          * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands

          * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.

          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs

          -----

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlFilter.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlLists.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControllerProtocol.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/Permission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/TablePermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/UserPermission.java PRE-CREATION

          security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/SecureTestUtil.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessControlFilter.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessController.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestTablePermissions.java PRE-CREATION

          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestZKPermissionsWatcher.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 46a1a3d

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 699a5f5

          src/main/resources/hbase-default.xml 2c8f44b

          src/main/ruby/hbase.rb 4d27191

          src/main/ruby/hbase/admin.rb b244ffe

          src/main/ruby/hbase/hbase.rb beb2450

          src/main/ruby/hbase/security.rb PRE-CREATION

          src/main/ruby/shell.rb 9a47600

          src/main/ruby/shell/commands.rb a352c2e

          src/main/ruby/shell/commands/grant.rb PRE-CREATION

          src/main/ruby/shell/commands/revoke.rb PRE-CREATION

          src/main/ruby/shell/commands/table_permission.rb PRE-CREATION

          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java 4d7ee22

          Diff: https://reviews.apache.org/r/2041/diff

          Testing

          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/#review2077 ----------------------------------------------------------- Looks good. The majority of my comments have to do with inconsistent logging practice. security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlFilter.java < https://reviews.apache.org/r/2041/#comment4718 > Could be stated better. security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlLists.java < https://reviews.apache.org/r/2041/#comment4719 > No. security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlLists.java < https://reviews.apache.org/r/2041/#comment4720 > Comment needs updating. security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java < https://reviews.apache.org/r/2041/#comment4721 > Can we make this 1? security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java < https://reviews.apache.org/r/2041/#comment4775 > Debug logging should go to LOG not AUDITLOG security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java < https://reviews.apache.org/r/2041/#comment4782 > Should be INFO or TRACE level? TRACE makes more sense to me. security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java < https://reviews.apache.org/r/2041/#comment4776 > Debug logging should go to LOG not AUDITLOG security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java < https://reviews.apache.org/r/2041/#comment4779 > Should be INFO or TRACE level? TRACE makes more sense to me. security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java < https://reviews.apache.org/r/2041/#comment4795 > Should something go to AUDITLOG here? security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java < https://reviews.apache.org/r/2041/#comment4797 > Should hasFamilyQualifierPermission log to AUDITLOG? It is used in places to make decisions – an exception is thrown directly or not. security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java < https://reviews.apache.org/r/2041/#comment4783 > Another one of these was sent to AUDITLOG above. Do the same here? Should be INFO or TRACE level? TRACE makes more sense to me. security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java < https://reviews.apache.org/r/2041/#comment4786 > Ultimately users should be allowed to enable or disable their own tables, but only after such operations don't carry as much systemic risk as they do currently. In that case, CREATE permission and an ownership check could follow the test for ADMIN permission. security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java < https://reviews.apache.org/r/2041/#comment4787 > As above security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java < https://reviews.apache.org/r/2041/#comment4791 > Should be logged with ERROR? security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java < https://reviews.apache.org/r/2041/#comment4799 > Would it be clearer then to call permissionGranted() something like hasColumnsPermission() ? Just a random thought. security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java < https://reviews.apache.org/r/2041/#comment4803 > Should this go to AUDITLOG? At INFO or TRACE level? My preference is TRACE. security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java < https://reviews.apache.org/r/2041/#comment4804 > Should this go to AUDITLOG? At INFO or TRACE level? My preference is TRACE. security/src/main/java/org/apache/hadoop/hbase/security/rbac/Permission.java < https://reviews.apache.org/r/2041/#comment4807 > What if instead we check for version 0 and throw an IllegalArgumentException if so? Technically, it is an invalid request if it contains an unrecognizable action code. Skipping this check if version > 0 would be a way to handle new perms while not accepting incorrect input otherwise. security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java < https://reviews.apache.org/r/2041/#comment4813 > Maybe we can call this ".auth."? We don't really have an RBAC implementation yet. Likewise for the package name for all of this stuff? Just a random thought. security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java < https://reviews.apache.org/r/2041/#comment4815 > Isn't this an error? security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java < https://reviews.apache.org/r/2041/#comment4816 > Should be at DEBUG level security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java < https://reviews.apache.org/r/2041/#comment4814 > I wonder if there is some way we can check if a secure variant of ZooKeeper is running, and refuse to initialize if not. security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java < https://reviews.apache.org/r/2041/#comment4823 > Shouldn't we propagate ZK exceptions upward? or at least convert them to IOE and throw those? Otherwise the permission cache is silently at risk of being out of sync with the ACL table. The safest thing to do is force a region close by bubbling up an exception from the coprocessor. This assumes that the coprocessor framework or regionserver will trigger a region close if it receives an unhandled exception from coprocessor code, and that this won't down the whole regionserver. security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java < https://reviews.apache.org/r/2041/#comment4825 > As above security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java < https://reviews.apache.org/r/2041/#comment4826 > As above security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java < https://reviews.apache.org/r/2041/#comment4827 > As above src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java < https://reviews.apache.org/r/2041/#comment4829 > Yes I agree this makes sense here, for convenience in setting ownership through the existing alter functionality. Andrew On 2011-09-23 19:14:20, Gary Helmling wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- (Updated 2011-09-23 19:14:20) Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: * AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. * AccessController - - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. - implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. * ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. * Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands * Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs ----- security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 46a1a3d src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 699a5f5 src/main/resources/hbase-default.xml 2c8f44b src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb b244ffe src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/table_permission.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java 4d7ee22 Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2041/
          -----------------------------------------------------------

          Review request for hbase.

          Summary
          -------

          This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine).

          Key parts of the implementation are:

          • AccessControlLists - encapsulates storage of permission grants in a metadata table ("acl"). This differs from previous implementation where the ".META." table was used to store permissions.
          • AccessController -
          • implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown.
          • implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions.
          • ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename.
          • Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands
          • Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier.

          This addresses bug HBASE-3025.
          https://issues.apache.org/jira/browse/HBASE-3025

          Diffs


          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlFilter.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlLists.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControllerProtocol.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/rbac/Permission.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/rbac/TablePermission.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/rbac/UserPermission.java PRE-CREATION
          security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/rbac/SecureTestUtil.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessControlFilter.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessController.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestTablePermissions.java PRE-CREATION
          security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestZKPermissionsWatcher.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 46a1a3d
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 699a5f5
          src/main/resources/hbase-default.xml 2c8f44b
          src/main/ruby/hbase.rb 4d27191
          src/main/ruby/hbase/admin.rb b244ffe
          src/main/ruby/hbase/hbase.rb beb2450
          src/main/ruby/hbase/security.rb PRE-CREATION
          src/main/ruby/shell.rb 9a47600
          src/main/ruby/shell/commands.rb a352c2e
          src/main/ruby/shell/commands/grant.rb PRE-CREATION
          src/main/ruby/shell/commands/revoke.rb PRE-CREATION
          src/main/ruby/shell/commands/table_permission.rb PRE-CREATION
          src/main/ruby/shell/commands/user_permission.rb PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java 4d7ee22

          Diff: https://reviews.apache.org/r/2041/diff

          Testing
          -------

          Thanks,

          Gary

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2041/ ----------------------------------------------------------- Review request for hbase. Summary ------- This patch implements access control list based authorization of HBase operations. The patch depends on the currently posted patch for HBASE-2742 (secure RPC engine). Key parts of the implementation are: AccessControlLists - encapsulates storage of permission grants in a metadata table (" acl "). This differs from previous implementation where the ".META." table was used to store permissions. AccessController - implements MasterObserver and RegionObserver, performing authorization checks in each of the preXXX() hooks. If authorization fails, an AccessDeniedException is thrown. implements AccessControllerProtocol as a coprocessor endpoint to provide RPC methods for granting, revoking and listing permissions. ZKPermissionWatcher (and TableAuthManager) - synchronizes ACL entries and updates throughout the cluster nodes using ZK. ACL entries are stored in per-table znodes as /hbase/acl/tablename. Additional ruby shell scripts providing the "grant", "revoke" and "user_permission" commands Support for a new OWNER attribute in HTableDescriptor. I could separate out this change into a new JIRA for discussion, but I don't see it as currently useful outside of security. Alternately, I could handle the OWNER attribute completely in AccessController without changing HTD, but that would make interaction via hbase shell a bit uglier. This addresses bug HBASE-3025 . https://issues.apache.org/jira/browse/HBASE-3025 Diffs security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlFilter.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControlLists.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessController.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/AccessControllerProtocol.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/Permission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/TableAuthManager.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/TablePermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/UserPermission.java PRE-CREATION security/src/main/java/org/apache/hadoop/hbase/security/rbac/ZKPermissionWatcher.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/SecureTestUtil.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessControlFilter.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestAccessController.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestTablePermissions.java PRE-CREATION security/src/test/java/org/apache/hadoop/hbase/security/rbac/TestZKPermissionsWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 46a1a3d src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 699a5f5 src/main/resources/hbase-default.xml 2c8f44b src/main/ruby/hbase.rb 4d27191 src/main/ruby/hbase/admin.rb b244ffe src/main/ruby/hbase/hbase.rb beb2450 src/main/ruby/hbase/security.rb PRE-CREATION src/main/ruby/shell.rb 9a47600 src/main/ruby/shell/commands.rb a352c2e src/main/ruby/shell/commands/grant.rb PRE-CREATION src/main/ruby/shell/commands/revoke.rb PRE-CREATION src/main/ruby/shell/commands/table_permission.rb PRE-CREATION src/main/ruby/shell/commands/user_permission.rb PRE-CREATION src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java 4d7ee22 Diff: https://reviews.apache.org/r/2041/diff Testing ------- Thanks, Gary
          Hide
          Francis Liu added a comment -

          Hi, With this access control scheme, what is the recommended way of doing bulk imports? We are looking to have a secured version of hbase on our shared clusters and bulk import will be largely the method data will be fed into hbase.

          Show
          Francis Liu added a comment - Hi, With this access control scheme, what is the recommended way of doing bulk imports? We are looking to have a secured version of hbase on our shared clusters and bulk import will be largely the method data will be fed into hbase.
          Hide
          Andrew Purtell added a comment -

          This feature has three dependencies:

          1) Coprocessor framework – In HBase 0.92

          2) Secure RPC engine – Because we don't pass the user in current HBase RPC requests, not in HBase 0.92 (yet)

          3) Secure ZooKeeper - Integrated in ZooKeeper trunk as ZOOKEEPER-938

          If all the dependencies are available, this feature can be made available with a simple patch.

          Show
          Andrew Purtell added a comment - This feature has three dependencies: 1) Coprocessor framework – In HBase 0.92 2) Secure RPC engine – Because we don't pass the user in current HBase RPC requests, not in HBase 0.92 (yet) 3) Secure ZooKeeper - Integrated in ZooKeeper trunk as ZOOKEEPER-938 If all the dependencies are available, this feature can be made available with a simple patch.
          Hide
          Timo Saarinen added a comment -

          What are the plans and schedule related to this feature? Will it be included in 0.92?

          Show
          Timo Saarinen added a comment - What are the plans and schedule related to this feature? Will it be included in 0.92?
          Hide
          stack added a comment -

          Moving out of 0.92.0. Pull it back in if you think different.

          Show
          stack added a comment - Moving out of 0.92.0. Pull it back in if you think different.
          Hide
          Andrew Purtell added a comment -

          Bringing into 0.92.

          Show
          Andrew Purtell added a comment - Bringing into 0.92.
          Hide
          Gary Helmling added a comment -

          @Joey,

          See HBASE-2014 for audit logging. It is not yet implemented, but we will be getting to it in the near future.

          Show
          Gary Helmling added a comment - @Joey, See HBASE-2014 for audit logging. It is not yet implemented, but we will be getting to it in the near future.
          Hide
          Joey Hung added a comment -

          Hi Gary,

          Forgot to ask this other question:
          Can both user and program gain access to data in the current access control implementation?

          Show
          Joey Hung added a comment - Hi Gary, Forgot to ask this other question: Can both user and program gain access to data in the current access control implementation?
          Hide
          Joey Hung added a comment -

          Hi Gary,

          Thanks for your quick reply.
          One more question, do you know anyone implementing auditing log for HBase security?
          Seems the Hadoop security has already implemented such feature.

          Show
          Joey Hung added a comment - Hi Gary, Thanks for your quick reply. One more question, do you know anyone implementing auditing log for HBase security? Seems the Hadoop security has already implemented such feature.
          Hide
          stack added a comment -

          I was going to take a look at the patch then saw its 300k. Will be back later after the popcorn is done (smile). Good on you Gary.

          Show
          stack added a comment - I was going to take a look at the patch then saw its 300k. Will be back later after the popcorn is done (smile). Good on you Gary.
          Hide
          stack added a comment -

          I was going to take a look at the patch then saw its 300k. Will be back later after the popcorn is done (smile). Good on you Gary.

          Show
          stack added a comment - I was going to take a look at the patch then saw its 300k. Will be back later after the popcorn is done (smile). Good on you Gary.
          Hide
          Gary Helmling added a comment -

          Updated preview patch for security features. Incorporates proper checking of master operations by implementing MasterObserver.

          Show
          Gary Helmling added a comment - Updated preview patch for security features. Incorporates proper checking of master operations by implementing MasterObserver.
          Hide
          Gary Helmling added a comment -

          Authentication is currently limited to Kerberos (v5), same as Hadoop security. PKI (and SSL certificate) authentication is not supported.

          HBase RPC encryption is supported via a configuration parameter (with options for integrity or confidentiality). RPC connections support mutual authentication of endpoints.

          Authorization is the core feature of this issue. Authorization of data access and operations is performed based on access control lists configured per table or per column family. Once the client has authenticated with a region server, the client will be able to perform read/write operations on tables/column families to which the client is granted access, as defined by the access control lists. Attempts to read/write data which have not been allowed will throw an AccessDeniedException.

          Show
          Gary Helmling added a comment - Authentication is currently limited to Kerberos (v5), same as Hadoop security. PKI (and SSL certificate) authentication is not supported. HBase RPC encryption is supported via a configuration parameter (with options for integrity or confidentiality). RPC connections support mutual authentication of endpoints. Authorization is the core feature of this issue. Authorization of data access and operations is performed based on access control lists configured per table or per column family. Once the client has authenticated with a region server, the client will be able to perform read/write operations on tables/column families to which the client is granted access, as defined by the access control lists. Attempts to read/write data which have not been allowed will throw an AccessDeniedException.
          Hide
          Joey Hung added a comment -

          Do the current implementation support other types of securities such as Public/Private Key, RSA, Authorization, Session, Communication, Certification, and Data Integrity?

          Or, is it true that once the client entered his access password, he gains access to all the data that he is permitted to read/write.

          Show
          Joey Hung added a comment - Do the current implementation support other types of securities such as Public/Private Key, RSA, Authorization, Session, Communication, Certification, and Data Integrity? Or, is it true that once the client entered his access password, he gains access to all the data that he is permitted to read/write.
          Hide
          Andrew Purtell added a comment -

          Regarding ZooKeeper, access is not controlled currently in a HBase installation. We are considering developing a Kerberos auth plugin for ZooKeeper, at which point ACLs can be set on znodes containing table permissions to prevent unprivileged users from subverting access control.

          Show
          Andrew Purtell added a comment - Regarding ZooKeeper, access is not controlled currently in a HBase installation. We are considering developing a Kerberos auth plugin for ZooKeeper, at which point ACLs can be set on znodes containing table permissions to prevent unprivileged users from subverting access control.
          Hide
          Andrew Purtell added a comment -

          Regarding the Master, more work needs to be done to disallow a user from taking administrative actions on other user's tables, such as enable/disable/drop. We only made one modification to the Master as a suggestion for one design option. With coprocessors its easy to encapsulate access control related changes to regionserver function. However, the coprocessor framework is a regionserver only feature. We don't as of yet have a similar extension framework for the Master. Therefore we need to consider one, or follow through with defining a concept of ownership and restrictions on administrative actions that can be taken by owners vs non-owners.

          Show
          Andrew Purtell added a comment - Regarding the Master, more work needs to be done to disallow a user from taking administrative actions on other user's tables, such as enable/disable/drop. We only made one modification to the Master as a suggestion for one design option. With coprocessors its easy to encapsulate access control related changes to regionserver function. However, the coprocessor framework is a regionserver only feature. We don't as of yet have a similar extension framework for the Master. Therefore we need to consider one, or follow through with defining a concept of ownership and restrictions on administrative actions that can be taken by owners vs non-owners.
          Hide
          Andrew Purtell added a comment -

          Attached is a first cut at a coprocessor based access controller. It requires the patches pending for HBASE-2001 and HBASE-2002/HBASE-2321, and has additional dependency on secure Hadoop classes. Therefore we expect this to be developed in a feature branch until HBase compilation and operation on secure vs. nonsecure Hadoop flavors can be made seamless.

          Package documentation will be forthcoming.

          This access controller coprocessor should be associated with all tables as a system extension. (These are coprocessors listed in hbase-site.xml and loaded into the regionserver at an early time. See package docs in the patch on HBASE-2001 for more information pertaining to coprocessors specifically.)

          Permissions for user tables are stored in a new acl: family in META. The access controller is active on META and user tables. When active on META, the access controller mirrors the contents of this family into a znode tree in zookeeper and updates the mirrored permissions information when values in acl: are added, changed, or deleted. When active on user tables, the access controller reads the permissions for its table from the appropriate znode, caches them, and sets a watch on the znode, updating the local cache whenever the znode data changes. The acl: family in META serves as persistent storage for access policy and as the canonical interface for defining access permissions. ZooKeeper serves to immediately and atomically propagate policy changes into the local permissions caches of all nodes in the cluster. For the typical user operation neither META nor ZooKeeper need be consulted when determining if a user has sufficient access privilege; up to date information will be found in the local cache.

          A new shell command, grant, is added to support granting or revoking specific rights to tables. This is accomplished by puts or deletes into the new acl: family in META.

          When tables are created the current (creating) user is given ownership of it. A table owner has full access to the table and can grant additional access. Ownership is tracked using an attribute of HTableDescriptor. Therefore currently a table must be disabled and modified to change ownership.

          This patch also contains a small modification to HMaster that introduces the concept of superuser, a specially privileged principal defined via configuration variable, or by default the principal under which the Master or RegionServer processes are running. (Currently, for proper functioning of the cluster, the superuser must be the principal under which the processes are running.) Only the superuser can modify a table descriptor. This prevents a user from arbitrarily reassigning ownership and therefore bypassing access control. We may keep the superuser concept or replace it with an explicit ALTER permission. Perhaps even only the superuser should be allowed to enable and disable tables, though this patch does not at present prevent any user from taking those actions.

          Show
          Andrew Purtell added a comment - Attached is a first cut at a coprocessor based access controller. It requires the patches pending for HBASE-2001 and HBASE-2002 / HBASE-2321 , and has additional dependency on secure Hadoop classes. Therefore we expect this to be developed in a feature branch until HBase compilation and operation on secure vs. nonsecure Hadoop flavors can be made seamless. Package documentation will be forthcoming. This access controller coprocessor should be associated with all tables as a system extension. (These are coprocessors listed in hbase-site.xml and loaded into the regionserver at an early time. See package docs in the patch on HBASE-2001 for more information pertaining to coprocessors specifically.) Permissions for user tables are stored in a new acl: family in META. The access controller is active on META and user tables. When active on META, the access controller mirrors the contents of this family into a znode tree in zookeeper and updates the mirrored permissions information when values in acl: are added, changed, or deleted. When active on user tables, the access controller reads the permissions for its table from the appropriate znode, caches them, and sets a watch on the znode, updating the local cache whenever the znode data changes. The acl: family in META serves as persistent storage for access policy and as the canonical interface for defining access permissions. ZooKeeper serves to immediately and atomically propagate policy changes into the local permissions caches of all nodes in the cluster. For the typical user operation neither META nor ZooKeeper need be consulted when determining if a user has sufficient access privilege; up to date information will be found in the local cache. A new shell command, grant , is added to support granting or revoking specific rights to tables. This is accomplished by puts or deletes into the new acl: family in META. When tables are created the current (creating) user is given ownership of it. A table owner has full access to the table and can grant additional access. Ownership is tracked using an attribute of HTableDescriptor. Therefore currently a table must be disabled and modified to change ownership. This patch also contains a small modification to HMaster that introduces the concept of superuser , a specially privileged principal defined via configuration variable, or by default the principal under which the Master or RegionServer processes are running. (Currently, for proper functioning of the cluster, the superuser must be the principal under which the processes are running.) Only the superuser can modify a table descriptor. This prevents a user from arbitrarily reassigning ownership and therefore bypassing access control. We may keep the superuser concept or replace it with an explicit ALTER permission. Perhaps even only the superuser should be allowed to enable and disable tables, though this patch does not at present prevent any user from taking those actions.
          Hide
          Andrew Purtell added a comment -

          Update terminology as suggested by comments on HBASE-3045.

          Show
          Andrew Purtell added a comment - Update terminology as suggested by comments on HBASE-3045 .
          Hide
          Jeff Hammerbacher added a comment -

          For those who don't know, RBAC stands for "Role-Based Access Control"

          Show
          Jeff Hammerbacher added a comment - For those who don't know, RBAC stands for "Role-Based Access Control"
          Hide
          Eugene Koontz added a comment -

          s/to strong/too strong/

          Show
          Eugene Koontz added a comment - s/to strong/too strong/
          Hide
          Eugene Koontz added a comment -

          removed 'blocks'; seems to strong - but at least it's related to hbase-1697.

          Show
          Eugene Koontz added a comment - removed 'blocks'; seems to strong - but at least it's related to hbase-1697.

            People

            • Assignee:
              Unassigned
              Reporter:
              Andrew Purtell
            • Votes:
              0 Vote for this issue
              Watchers:
              21 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development