diff --git src/main/docbkx/security.xml src/main/docbkx/security.xml
index 4fe5aa0..2320653 100644
--- src/main/docbkx/security.xml
+++ src/main/docbkx/security.xml
@@ -338,7 +338,7 @@ grant 'rest_server', 'RWCA'
- Simple User Access to Apache HBase
+ Simple Authentication and AuthorizationNewer releases of Apache HBase (>= 0.92) support optional SASL authentication of clientsSee also Matteo Bertozzi's article on Understanding
@@ -467,8 +467,598 @@ grant 'rest_server', 'RWCA'
-
+
+ Securing Your Data
+ After you have configured secure authentication between HBase client and server processes
+ and gateways, you need to consider the security of your data itself. HBase provides several
+ strategies for securing your data:
+
+
+ Role-based Access Control (RBAC) based upon groups and permissions
+
+
+ Visibility Labels which allow you to label cells and control access to labelled
+ cells
+
+
+ Transparent encryption of data at rest and in the WAL
+
+
+ This section of the documentation will provide some definitions and conceptual
+ information, followed by more information about each of the above concerns, and ending in a
+ simplified real-world example of a security scenario and how it might be implemented in HBase.
+ Some notes about performance impacts of the different security choices are discussed
+ throughout.
+
+ HBase security is in active development and is evolving rapidly. Any strategy you employ
+ for security of your data should be thoroughly tested. In addition, some of these features
+ are still in the experimental stage of development. To take advantage of many of these
+ features, you must be running HBase 0.98+.
+
+
+
+ Concepts and Definitions
+
+ Tags
+
+ Tags are a feature of HFile v3. A tag is an piece of metadata
+ which is part of a cell, separate from the key, value, and version. Tags are an
+ implementation detail which provides a foundation for other security-related features
+ such as cell-level ACLs and visibility labels. Tags are stored in the HFiles
+ themselves.
+
+
+
+ Role-Based Access Control (RBAC)
+
+ Role-based access control (RBAC) is a familiar security concept, where users are
+ assigned to groups, and groups inherit sets of permissions for resources or classes of
+ resources. In HBase, permissions can be granted to roles at the level of the namespace,
+ table, column family, or cell. Cell-level ACLs are a new feature of HBase 0.98 and
+ newer. Grantable permissions in HBase include Read, Write, Execute (new in HBase 0.98),
+ Admin, and Create.
+
+
+
+ Visibility Labels
+
+ A visibility label is a type of tag, as described in .
+ The label itself is an arbitrary text string, and the set of labels available to a
+ cluster is curated and administered at the server level. Examples of possible visibility
+ labels are "topsecret" and "public". Visibility labels can be combined to form
+ expressions, using logical operators (|, &, !). Expressions are evaluated using
+ Boolean logic. A cell is labeled with a visibility label during a Put operation, and a
+ user's ability to see the cell is evaluated during the Get or Scan operation.
+
+
+
+ Data At Rest
+
+ Data at rest refers to HBase data which is persisted to HDFS or another filesystem
+ in an HFile or is present in memory in the WAL.
+
+
+
+ Transparent Encryption
+
+ With transparent encryption, data is encrypted before being written to persistent
+ storage, and is decrypted in order to be read from persistent storage, automatically
+ with no special action required by the client.
+
+
+
+
+
+ Basic Server-Side Configuration
+
+ Enable HFile v3, by setting to 3 in
+ hbase-site.xml. This is the default for HBase 0.98 and
+ newer.
+
+ hfile.format.version
+ 3
+
+ ]]>
+
+
+ Enable SASL and Kerberos authentication for RPC and ZooKeeper, as described in and .
+
+
+
+
+ Access Control Labels (ACLs)
+
+ How It Works
+ ACLs in HBase are based upon a user's membership in or exclusion from groups, and a
+ given group's permissions to access a given resource. ACLs are implemented as a
+ coprocessor called AccessController.
+ The users are and groups stored in a directory such as an LDAP or Active Directory
+ service. A Hadoop group mapper maps between directory groups and
+ HBase groups. Any supported Hadoop group mapper will work. Groups are then granted
+ specific permissions (Read, Write, Execute, Create, Admin) against resources (global,
+ namespaces, tables, cells, or endpoints).
+ ACLs can be used together with Visibility Labels.
+ In order to use cell-level ACLs, you must be using HFile v3 and HBase 0.98.
+
+
+ Server-Side Configuration
+
+
+ As a prerequisite, perform the steps in .
+
+ Install and configure the AccessController coprocessor, by setting the
+ following properties in hbase-site.xml. These properties take a
+ list of classes, and in practice, you will probably enable the VisibilityController as
+ well, but this configuration example only shows the settings for the
+ AccessibilityController.
+
+ hbase.coprocessor.region.classes
+ org.apache.hadoop.hbase.security.access.AccessController, org.apache.hadoop.hbase.security.token.TokenProvider
+
+
+ hbase.coprocessor.master.classes
+ org.apache.hadoop.hbase.security.access.AccessController
+
+
+ hbase.coprocessor.regionserver.classes
+ org.apache.hadoop/hbase.security.access.AccessController
+
+
+ hbase.security.exec.permission.checks
+ true
+
+ ]]>
+ Optionally, you can enable transport security, by setting
+ to auth-conf.
+
+
+ Set up the Hadoop group mapper in the Hadoop namenode's
+ core-site.xml. This is a Hadoop file, not an HBase file.
+ Customize it to your site's needs. Following is an example.
+
+ hadoop.security.group.mapping
+ org.apache.hadoop.security.LdapGroupsMapping
+
+
+
+ hadoop.security.group.mapping.ldap.url
+ ldap://server
+
+
+
+ hadoop.security.group.mapping.ldap.bind.user
+ Administrator@example-ad.local
+
+
+
+ hadoop.security.group.mapping.ldap.bind.password
+ ****
+
+
+
+ hadoop.security.group.mapping.ldap.base
+ dc=example-ad,dc=local
+
+
+
+ hadoop.security.group.mapping.ldap.search.filter.user
+ (&(objectClass=user)(sAMAccountName={0}))
+
+
+
+ hadoop.security.group.mapping.ldap.search.filter.group
+ (objectClass=group)
+
+
+
+ hadoop.security.group.mapping.ldap.search.attr.member
+ member
+
+
+
+ hadoop.security.group.mapping.ldap.search.attr.group.name
+ cn
+
+
+ ]]>
+
+
+ Distribute your confiuration and restart your cluster for changes to take
+ effect.
+
+
+ To test your configuration, log into HBase Shell as a given user and use the
+ whoami command to report the groups your user is part of. In this example, the user is
+ reported as being a member of the services group.
+
+hbase> whoami
+service (auth:KERBEROS)
+ groups: services
+
+
+
+
+
+ Administration
+ Administration tasks can be performed from HBase Shell or via an API.
+
+
+ User and Group Administration
+ Users and groups are maintained external to HBase, in your directory.
+
+
+ Granting Access To A Namespace, Table, Column Family, or Cell
+ There are a few different types of syntax for grant statements. The first, and
+ most familiar, is as follows, with the table and column family being optional:
+ grant 'user', 'RWXCA', 'TABLE', 'CF', 'CQ'
+ Groups and users are granted access in the same way, but groups are prefixed with
+ an @ symbol. In the same way, tables and namespaces are specified
+ in the same way, but namespacesa re prefixed with an @
+ symbol.
+ It is also possible to specify multiple grants against the same resource in a
+ single statement, as in this example. The first sub-clause maps users or groups to
+ ACLs and the second sub-clause specifies the resource.
+ The following statement grants permissions to the user table,
+ specifically cells in the pii column whose value starts with the string
+ test. The developers group is granted Read and Write, and the
+ testuser user is granted Read.
+ hbase> grant 'user', \
+ { '@developers' => 'RW', 'testuser' => 'R' }, \
+ { COLUMNS => 'pii', FILTER => "(PrefixFilter ('test'))" }
+
+ HBase Shell
+
+
+ Global:
+ hbase> grant '@admins', 'RWXCA'
+
+
+ Namespace:
+ hbase> grant 'service', 'RWXCA', '@test-NS'
+
+
+ Table:
+ hbase> grant 'service', 'RWXCA', 'user'
+
+
+ Column Family:
+ hbase> grant '@developers', 'RW', 'user', 'i'
+
+
+ Column Qualifier:
+ hbase> grant 'service, 'RW', 'user', 'i', 'foo'
+
+
+ Cell:
+ ?? I can't find this.
+
+
+
+
+ API
+ See the source files
+ hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
+ and
+ hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java
+ for examples of setting and checking ACLs using the Java API. The following example
+ shows how to grant access at the table level.
+ () {
+ @Override
+ public Void call() throws Exception {
+ HTable acl = new HTable(util.getConfiguration(), AccessControlLists.ACL_TABLE_NAME);
+ try {
+ BlockingRpcChannel service = acl.coprocessorService(HConstants.EMPTY_START_ROW);
+ AccessControlService.BlockingInterface protocol =
+ AccessControlService.newBlockingStub(service);
+ ProtobufUtil.grant(protocol, user, table, family, qualifier, actions);
+ } finally {
+ acl.close();
+ }
+ return null;
+ }
+ });
+} ]]>
+
+
+
+
+ Revoking Access Control From a Namespace, Table, Column Family, or Cell
+ The revoke command and API are twins of the grant command and
+ API, and the syntax is exactly the same. You can only revoke access that has
+ previously been granted. A revoke statement is not the same thing
+ as explicit denial to a resource.
+
+ Revoking Access To a Table
+
+() {
+ @Override
+ public Void call() throws Exception {
+ HTable acl = new HTable(util.getConfiguration(), AccessControlLists.ACL_TABLE_NAME);
+ try {
+ BlockingRpcChannel service = acl.coprocessorService(HConstants.EMPTY_START_ROW);
+ AccessControlService.BlockingInterface protocol =
+ AccessControlService.newBlockingStub(service);
+ ProtobufUtil.revoke(protocol, user, table, family, qualifier, actions);
+ } finally {
+ acl.close();
+ }
+ return null;
+ }
+ });
+} ]]>
+
+
+
+
+ Show a User's Effective Permissions
+
+ HBase Shell
+ hbase> user_permission 'user'
+ hbase> user_permission '.*'
+ hbase> user_permission JAVA_REGEX
+
+
+ API
+ ) {
+ List> results = (List>) obj;
+ if (results != null && results.isEmpty()) {
+ fail("Empty non null results from action for user '" + user.getShortName() + "'");
+ }
+ assertEquals(count, results.size());
+ }
+ } catch (AccessDeniedException ade) {
+ fail("Expected action to pass for user '" + user.getShortName() + "' but was denied");
+ }
+} ]]>
+
+
+
+
+
+
+
+
+ Visibility Labels
+ Visibility labels control can be used to only permit users or principals associated with
+ a given label to read or access cells with that label. For instance, you might label a cell
+ top-secret, and only grant access to that label to the
+ managers group. Visibility labels are implemented using Tags, which are
+ a feature of HFile v3, and allow you to store metadata on a per-cell basis. A label is a
+ string, and labels can be combined into expressions by using logical operators (&, |, or
+ !), and using parentheses for grouping. Examples will be provided below.
+ If a user or group's labels do not match a cell's label or expression, the user is
+ denied access to the cell.
+ A user adds visibility labels to a cell during a Put operation The user does not need to
+ access to a label in order to label cells with it. Whether a user is authorized to read a
+ labelled cell is determined during a Get or Scan, and results which the user is not allowed
+ to read are filtered out. This incurs the same IO penalty as if the results were returned,
+ but reduces load on the network. Visibility label access checking is performed by the
+ VisibilityController coprocessor endpoint executor. Visibility labels can be used in
+ conjunction with ACLs.
+ Visibility label expressions are evaluated from left to right, using parentheses to
+ group clauses. HBase does not check to see whether the expression makes logical or semantic
+ sense.
+
+ Server-Side Configuration
+
+
+ As a prerequisite, perform the steps in .
+
+ Install and configure the VisibilityController coprocessor, by setting the
+ following properties in hbase-site.xml. These properties take a
+ list of classes, and in practice, you will probably enable the AccessController as
+ well, but this configuration example only shows the settings for the
+ VisibilityController.
+
+ hbase.coprocessor.region.classes
+ org.apache.hadoop.hbase.security.visibility.VisibilityController, org.apache.hadoop.hbase.security.token.TokenProvider
+
+
+ hbase.coprocessor.master.classes
+ org.apache.hadoop.hbase.security.visibility.VisibilityController
+
+
+ hbase.coprocessor.regionserver.classes
+ org.apache.hadoop.hbase.security.access.AccessController
+
+
+ hbase.security.exec.permission.checks
+ true
+
+ ]]>
+ Optionally, you can enable transport security, by setting
+ to auth-conf.
+
+
+ Adjust Configuration
+ By default, users can label cells with labels they do not have access to, which
+ means that a user can Put data that he cannot read.. If you only want users to be able
+ to label cells with labels they do have access to, , set
+ hbase.security.visibility.mutations.checkauths to
+ true.
+
+
+ Distribute your confiuration and restart your cluster for changes to take
+ effect.
+
+
+
+
+ Administration
+ Administration tasks can be performed using the HBase Shell or the Admin API. For
+ defining the list of visibility labels and associating labels with users or groups, the
+ HBase Shell is probably simpler.
+
+
+ Define the List of Visibility Labels
+
+ HBase Shell
+ hbase< add_labels [ 'admin', 'service', 'developer', 'test' ]
+
+
+ Java API
+ This example was taken from the source file
+ hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabels.java.
+ Refer to that file or the API documentation for more context.
+ action =
+ new PrivilegedExceptionAction() {
+ public VisibilityLabelsResponse run() throws Exception {
+ String[] labels = { SECRET, TOPSECRET, CONFIDENTIAL, PUBLIC, PRIVATE, COPYRIGHT, ACCENT,
+ UNICODE_VIS_TAG, UC1, UC2 };
+ try {
+ VisibilityClient.addLabels(conf, labels);
+ } catch (Throwable t) {
+ throw new IOException(t);
+ }
+ return null;
+ }
+ };
+ SUPERUSER.runAs(action);
+}
+ ]]>
+
+
+
+ Associate Labels with Users or Groups
+
+ HBase Shell
+ hbase< set_auths 'service', [ 'service' ]
+ hbase< set_auths 'testuser', [ 'test' ]
+ hbase< set_auths 'qa', [ 'test', 'developer' ]
+
+
+ Java API
+ This example was taken from the source file
+ hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabels.java.
+ Refer to that file or the API documentation for more context.
+ action = new PrivilegedExceptionAction() {
+ public Void run() throws Exception {
+ String[] auths = { SECRET, CONFIDENTIAL };
+ try {
+ VisibilityClient.setAuths(conf, auths, user);
+ } catch (Throwable e) {
+ }
+ return null;
+ }
+ ...
+ ]]>
+
+
+
+ Clear Labels From Users or Groups
+
+ HBase Shell
+ hbase< clear_auths 'service', [ 'service' ]
+ hbase< clear_auths 'testuser', [ 'test' ]
+ hbase< clear_auths 'qa', [ 'test', 'developer' ]
+
+
+ Java API
+ This example was taken from the source file
+ hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabels.java.
+ Refer to that file or the API documentation for more context.
+
+
+
+
+ Apply a Label or Expression to a Cell
+ The label is only applied when data is written. The label is associated with a
+ given version of the cell.
+
+ HBase Shell
+ hbase< set_visibility 'user', 'admin|service|developer', \
+ { COLUMNS => 'i' }
+ hbase< set_visibility 'user', 'admin|service', \
+ { COLUMNS => ' pii' }
+ hbase< COLUMNS => [ 'i', 'pii' ], \
+ FILTER => "(PrefixFilter ('test'))" }
+
+
+ Java API
+ This example was taken from the source file
+ hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabels.java.
+ Refer to that file or the API documentation for more context.
+ puts = new ArrayList();
+ for (String labelExp : labelExps) {
+ Put put = new Put(Bytes.toBytes("row" + i));
+ put.add(fam, qual, HConstants.LATEST_TIMESTAMP, value);
+ put.setCellVisibility(new CellVisibility(labelExp));
+ puts.add(put);
+ i++;
+ }
+ table.put(puts);
+ } finally {
+ if (table != null) {
+ table.flushCommits();
+ }
+ }
+ ]]>
+
+
+
+
+
+
+
+ Transparent Encryption of Data
+
+ Server-Side Configuration
+
+
+
+ Administration
+
+
+
+
+
+ Data Security Example
+
+
+
+
+
+ Tags Every cell can have metadata associated with it. Adding metadata in the data part of
every cell would make things difficult.