diff --git src/main/docbkx/security.xml src/main/docbkx/security.xml index 4fe5aa0..2320653 100644 --- src/main/docbkx/security.xml +++ src/main/docbkx/security.xml @@ -338,7 +338,7 @@ grant 'rest_server', 'RWCA'
- Simple User Access to Apache HBase + Simple Authentication and Authorization Newer releases of Apache HBase (>= 0.92) support optional SASL authentication of clients See also Matteo Bertozzi's article on Understanding @@ -467,8 +467,598 @@ grant 'rest_server', 'RWCA'
-
+
+ Securing Your Data + After you have configured secure authentication between HBase client and server processes + and gateways, you need to consider the security of your data itself. HBase provides several + strategies for securing your data: + + + Role-based Access Control (RBAC) based upon groups and permissions + + + Visibility Labels which allow you to label cells and control access to labelled + cells + + + Transparent encryption of data at rest and in the WAL + + + This section of the documentation will provide some definitions and conceptual + information, followed by more information about each of the above concerns, and ending in a + simplified real-world example of a security scenario and how it might be implemented in HBase. + Some notes about performance impacts of the different security choices are discussed + throughout. + + HBase security is in active development and is evolving rapidly. Any strategy you employ + for security of your data should be thoroughly tested. In addition, some of these features + are still in the experimental stage of development. To take advantage of many of these + features, you must be running HBase 0.98+. + + + + Concepts and Definitions + + Tags + + Tags are a feature of HFile v3. A tag is an piece of metadata + which is part of a cell, separate from the key, value, and version. Tags are an + implementation detail which provides a foundation for other security-related features + such as cell-level ACLs and visibility labels. Tags are stored in the HFiles + themselves. + + + + Role-Based Access Control (RBAC) + + Role-based access control (RBAC) is a familiar security concept, where users are + assigned to groups, and groups inherit sets of permissions for resources or classes of + resources. In HBase, permissions can be granted to roles at the level of the namespace, + table, column family, or cell. Cell-level ACLs are a new feature of HBase 0.98 and + newer. Grantable permissions in HBase include Read, Write, Execute (new in HBase 0.98), + Admin, and Create. + + + + Visibility Labels + + A visibility label is a type of tag, as described in . + The label itself is an arbitrary text string, and the set of labels available to a + cluster is curated and administered at the server level. Examples of possible visibility + labels are "topsecret" and "public". Visibility labels can be combined to form + expressions, using logical operators (|, &, !). Expressions are evaluated using + Boolean logic. A cell is labeled with a visibility label during a Put operation, and a + user's ability to see the cell is evaluated during the Get or Scan operation. + + + + Data At Rest + + Data at rest refers to HBase data which is persisted to HDFS or another filesystem + in an HFile or is present in memory in the WAL. + + + + Transparent Encryption + + With transparent encryption, data is encrypted before being written to persistent + storage, and is decrypted in order to be read from persistent storage, automatically + with no special action required by the client. + + + + + + Basic Server-Side Configuration + + Enable HFile v3, by setting to 3 in + hbase-site.xml. This is the default for HBase 0.98 and + newer. + + hfile.format.version + 3 + + ]]> + + + Enable SASL and Kerberos authentication for RPC and ZooKeeper, as described in and . + + + +
+ Access Control Labels (ACLs) +
+ How It Works + ACLs in HBase are based upon a user's membership in or exclusion from groups, and a + given group's permissions to access a given resource. ACLs are implemented as a + coprocessor called AccessController. + The users are and groups stored in a directory such as an LDAP or Active Directory + service. A Hadoop group mapper maps between directory groups and + HBase groups. Any supported Hadoop group mapper will work. Groups are then granted + specific permissions (Read, Write, Execute, Create, Admin) against resources (global, + namespaces, tables, cells, or endpoints). + ACLs can be used together with Visibility Labels. + In order to use cell-level ACLs, you must be using HFile v3 and HBase 0.98. +
+
+ Server-Side Configuration + + + As a prerequisite, perform the steps in . + + Install and configure the AccessController coprocessor, by setting the + following properties in hbase-site.xml. These properties take a + list of classes, and in practice, you will probably enable the VisibilityController as + well, but this configuration example only shows the settings for the + AccessibilityController. + + hbase.coprocessor.region.classes + org.apache.hadoop.hbase.security.access.AccessController, org.apache.hadoop.hbase.security.token.TokenProvider + + + hbase.coprocessor.master.classes + org.apache.hadoop.hbase.security.access.AccessController + + + hbase.coprocessor.regionserver.classes + org.apache.hadoop/hbase.security.access.AccessController + + + hbase.security.exec.permission.checks + true + + ]]> + Optionally, you can enable transport security, by setting + to auth-conf. + + + Set up the Hadoop group mapper in the Hadoop namenode's + core-site.xml. This is a Hadoop file, not an HBase file. + Customize it to your site's needs. Following is an example. + + hadoop.security.group.mapping + org.apache.hadoop.security.LdapGroupsMapping + + + + hadoop.security.group.mapping.ldap.url + ldap://server + + + + hadoop.security.group.mapping.ldap.bind.user + Administrator@example-ad.local + + + + hadoop.security.group.mapping.ldap.bind.password + **** + + + + hadoop.security.group.mapping.ldap.base + dc=example-ad,dc=local + + + + hadoop.security.group.mapping.ldap.search.filter.user + (&(objectClass=user)(sAMAccountName={0})) + + + + hadoop.security.group.mapping.ldap.search.filter.group + (objectClass=group) + + + + hadoop.security.group.mapping.ldap.search.attr.member + member + + + + hadoop.security.group.mapping.ldap.search.attr.group.name + cn + + + ]]> + + + Distribute your confiuration and restart your cluster for changes to take + effect. + + + To test your configuration, log into HBase Shell as a given user and use the + whoami command to report the groups your user is part of. In this example, the user is + reported as being a member of the services group. + +hbase> whoami +service (auth:KERBEROS) + groups: services + + + +
+
+ Administration + Administration tasks can be performed from HBase Shell or via an API. + + + User and Group Administration + Users and groups are maintained external to HBase, in your directory. + + + Granting Access To A Namespace, Table, Column Family, or Cell + There are a few different types of syntax for grant statements. The first, and + most familiar, is as follows, with the table and column family being optional: + grant 'user', 'RWXCA', 'TABLE', 'CF', 'CQ' + Groups and users are granted access in the same way, but groups are prefixed with + an @ symbol. In the same way, tables and namespaces are specified + in the same way, but namespacesa re prefixed with an @ + symbol. + It is also possible to specify multiple grants against the same resource in a + single statement, as in this example. The first sub-clause maps users or groups to + ACLs and the second sub-clause specifies the resource. + The following statement grants permissions to the user table, + specifically cells in the pii column whose value starts with the string + test. The developers group is granted Read and Write, and the + testuser user is granted Read. + hbase> grant 'user', \ + { '@developers' => 'RW', 'testuser' => 'R' }, \ + { COLUMNS => 'pii', FILTER => "(PrefixFilter ('test'))" } + + HBase Shell + + + Global: + hbase> grant '@admins', 'RWXCA' + + + Namespace: + hbase> grant 'service', 'RWXCA', '@test-NS' + + + Table: + hbase> grant 'service', 'RWXCA', 'user' + + + Column Family: + hbase> grant '@developers', 'RW', 'user', 'i' + + + Column Qualifier: + hbase> grant 'service, 'RW', 'user', 'i', 'foo' + + + Cell: + ?? I can't find this. + + + + + API + See the source files + hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java + and + hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java + for examples of setting and checking ACLs using the Java API. The following example + shows how to grant access at the table level. + () { + @Override + public Void call() throws Exception { + HTable acl = new HTable(util.getConfiguration(), AccessControlLists.ACL_TABLE_NAME); + try { + BlockingRpcChannel service = acl.coprocessorService(HConstants.EMPTY_START_ROW); + AccessControlService.BlockingInterface protocol = + AccessControlService.newBlockingStub(service); + ProtobufUtil.grant(protocol, user, table, family, qualifier, actions); + } finally { + acl.close(); + } + return null; + } + }); +} ]]> + + + + + Revoking Access Control From a Namespace, Table, Column Family, or Cell + The revoke command and API are twins of the grant command and + API, and the syntax is exactly the same. You can only revoke access that has + previously been granted. A revoke statement is not the same thing + as explicit denial to a resource. + + Revoking Access To a Table + +() { + @Override + public Void call() throws Exception { + HTable acl = new HTable(util.getConfiguration(), AccessControlLists.ACL_TABLE_NAME); + try { + BlockingRpcChannel service = acl.coprocessorService(HConstants.EMPTY_START_ROW); + AccessControlService.BlockingInterface protocol = + AccessControlService.newBlockingStub(service); + ProtobufUtil.revoke(protocol, user, table, family, qualifier, actions); + } finally { + acl.close(); + } + return null; + } + }); +} ]]> + + + + + Show a User's Effective Permissions + + HBase Shell + hbase> user_permission 'user' + hbase> user_permission '.*' + hbase> user_permission JAVA_REGEX + + + API + ) { + List results = (List) obj; + if (results != null && results.isEmpty()) { + fail("Empty non null results from action for user '" + user.getShortName() + "'"); + } + assertEquals(count, results.size()); + } + } catch (AccessDeniedException ade) { + fail("Expected action to pass for user '" + user.getShortName() + "' but was denied"); + } +} ]]> + + + + +
+
+ +
+ Visibility Labels + Visibility labels control can be used to only permit users or principals associated with + a given label to read or access cells with that label. For instance, you might label a cell + top-secret, and only grant access to that label to the + managers group. Visibility labels are implemented using Tags, which are + a feature of HFile v3, and allow you to store metadata on a per-cell basis. A label is a + string, and labels can be combined into expressions by using logical operators (&, |, or + !), and using parentheses for grouping. Examples will be provided below. + If a user or group's labels do not match a cell's label or expression, the user is + denied access to the cell. + A user adds visibility labels to a cell during a Put operation The user does not need to + access to a label in order to label cells with it. Whether a user is authorized to read a + labelled cell is determined during a Get or Scan, and results which the user is not allowed + to read are filtered out. This incurs the same IO penalty as if the results were returned, + but reduces load on the network. Visibility label access checking is performed by the + VisibilityController coprocessor endpoint executor. Visibility labels can be used in + conjunction with ACLs. + Visibility label expressions are evaluated from left to right, using parentheses to + group clauses. HBase does not check to see whether the expression makes logical or semantic + sense. +
+ Server-Side Configuration + + + As a prerequisite, perform the steps in . + + Install and configure the VisibilityController coprocessor, by setting the + following properties in hbase-site.xml. These properties take a + list of classes, and in practice, you will probably enable the AccessController as + well, but this configuration example only shows the settings for the + VisibilityController. + + hbase.coprocessor.region.classes + org.apache.hadoop.hbase.security.visibility.VisibilityController, org.apache.hadoop.hbase.security.token.TokenProvider + + + hbase.coprocessor.master.classes + org.apache.hadoop.hbase.security.visibility.VisibilityController + + + hbase.coprocessor.regionserver.classes + org.apache.hadoop.hbase.security.access.AccessController + + + hbase.security.exec.permission.checks + true + + ]]> + Optionally, you can enable transport security, by setting + to auth-conf. + + + Adjust Configuration + By default, users can label cells with labels they do not have access to, which + means that a user can Put data that he cannot read.. If you only want users to be able + to label cells with labels they do have access to, , set + hbase.security.visibility.mutations.checkauths to + true. + + + Distribute your confiuration and restart your cluster for changes to take + effect. + + +
+
+ Administration + Administration tasks can be performed using the HBase Shell or the Admin API. For + defining the list of visibility labels and associating labels with users or groups, the + HBase Shell is probably simpler. + + + Define the List of Visibility Labels + + HBase Shell + hbase< add_labels [ 'admin', 'service', 'developer', 'test' ] + + + Java API + This example was taken from the source file + hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabels.java. + Refer to that file or the API documentation for more context. + action = + new PrivilegedExceptionAction() { + public VisibilityLabelsResponse run() throws Exception { + String[] labels = { SECRET, TOPSECRET, CONFIDENTIAL, PUBLIC, PRIVATE, COPYRIGHT, ACCENT, + UNICODE_VIS_TAG, UC1, UC2 }; + try { + VisibilityClient.addLabels(conf, labels); + } catch (Throwable t) { + throw new IOException(t); + } + return null; + } + }; + SUPERUSER.runAs(action); +} + ]]> + + + + Associate Labels with Users or Groups + + HBase Shell + hbase< set_auths 'service', [ 'service' ] + hbase< set_auths 'testuser', [ 'test' ] + hbase< set_auths 'qa', [ 'test', 'developer' ] + + + Java API + This example was taken from the source file + hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabels.java. + Refer to that file or the API documentation for more context. + action = new PrivilegedExceptionAction() { + public Void run() throws Exception { + String[] auths = { SECRET, CONFIDENTIAL }; + try { + VisibilityClient.setAuths(conf, auths, user); + } catch (Throwable e) { + } + return null; + } + ... + ]]> + + + + Clear Labels From Users or Groups + + HBase Shell + hbase< clear_auths 'service', [ 'service' ] + hbase< clear_auths 'testuser', [ 'test' ] + hbase< clear_auths 'qa', [ 'test', 'developer' ] + + + Java API + This example was taken from the source file + hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabels.java. + Refer to that file or the API documentation for more context. + + + + + Apply a Label or Expression to a Cell + The label is only applied when data is written. The label is associated with a + given version of the cell. + + HBase Shell + hbase< set_visibility 'user', 'admin|service|developer', \ + { COLUMNS => 'i' } + hbase< set_visibility 'user', 'admin|service', \ + { COLUMNS => ' pii' } + hbase< COLUMNS => [ 'i', 'pii' ], \ + FILTER => "(PrefixFilter ('test'))" } + + + Java API + This example was taken from the source file + hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabels.java. + Refer to that file or the API documentation for more context. + puts = new ArrayList(); + for (String labelExp : labelExps) { + Put put = new Put(Bytes.toBytes("row" + i)); + put.add(fam, qual, HConstants.LATEST_TIMESTAMP, value); + put.setCellVisibility(new CellVisibility(labelExp)); + puts.add(put); + i++; + } + table.put(puts); + } finally { + if (table != null) { + table.flushCommits(); + } + } + ]]> + + + +
+
+ +
+ Transparent Encryption of Data +
+ Server-Side Configuration + +
+
+ Administration + +
+
+ +
+ Data Security Example + +
+ +
+ +
Tags Every cell can have metadata associated with it. Adding metadata in the data part of every cell would make things difficult.