HBase
  1. HBase
  2. HBASE-2418

add support for ZooKeeper authentication

    Details

    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      This adds support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

      SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

        Server {
          com.sun.security.auth.module.Krb5LoginModule required
          useKeyTab=true
          keyTab="/etc/hbase/conf/hbase.keytab"
          storeKey=true
          useTicketCache=false
          principal="zookeeper/$HOSTNAME";
        };
        Client {
          com.sun.security.auth.module.Krb5LoginModule required
          useKeyTab=true
          useTicketCache=false
          keyTab="/etc/hbase/conf/hbase.keytab"
          principal="hbase/$HOSTNAME";
        };

      and then configure both the client and server processes to use it, for example in hbase-site.xml:

        HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
        HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"
        HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

      HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

      We will pull in a Hadoop artifact patched with HADOOP-7070 if building under the security profile (-P security). 0.20.205 does not yet include HADOOP-7070. Without it, the JAAS configuration required for secure operation of the ZooKeeper client will be ignored.
      Show
      This adds support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:   Server {     com.sun.security.auth.module.Krb5LoginModule required     useKeyTab=true     keyTab="/etc/hbase/conf/hbase.keytab"     storeKey=true     useTicketCache=false     principal="zookeeper/$HOSTNAME";   };   Client {     com.sun.security.auth.module.Krb5LoginModule required     useKeyTab=true     useTicketCache=false     keyTab="/etc/hbase/conf/hbase.keytab"     principal="hbase/$HOSTNAME";   }; and then configure both the client and server processes to use it, for example in hbase-site.xml:   HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"   HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"   HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. We will pull in a Hadoop artifact patched with HADOOP-7070 if building under the security profile (-P security). 0.20.205 does not yet include HADOOP-7070 . Without it, the JAAS configuration required for secure operation of the ZooKeeper client will be ignored.

      Description

      Some users may run a ZooKeeper cluster in "multi tenant mode" meaning that more than one client service would
      like to share a single ZooKeeper service instance (cluster). In this case the client services typically want to protect
      their data (ZK znodes) from access by other services (tenants) on the cluster. Say you are running HBase and Solr
      and Neo4j, or multiple HBase instances, etc... having authentication/authorization on the znodes is important for both
      security and helping to ensure that services don't interact negatively (touch each other's data).

      Today HBase does not have support for authentication or authorization. This should be added to the HBase clients
      that are accessing the ZK cluster. In general it means calling addAuthInfo once after a session is established:

      http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooKeeper.html#addAuthInfo(java.lang.String, byte[])

      with a user specific credential, often times this is a shared secret or certificate. You may be able to statically configure this
      in some cases (config string or file to read from), however in my case in particular you may need to access it programmatically,
      which adds complexity as the end user may need to load code into HBase for accessing the credential.

      Secondly you need to specify a non "world" ACL when interacting with znodes (create primarily):
      http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/data/ACL.html
      http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooDefs.html

      Feel free to ping the ZooKeeper team if you have questions. It might also be good to discuss with some
      potential end users - in particular regarding how the end user can specify the credential.

      1. 2418.addendum
        0.6 kB
        Ted Yu
      2. HBASE-2418-6.patch
        25 kB
        Andrew Purtell
      3. HBASE-2418-6.patch
        25 kB
        Andrew Purtell

        Issue Links

          Activity

          Patrick Hunt created issue -
          Andrew Purtell made changes -
          Field Original Value New Value
          Link This issue relates to HBASE-1697 [ HBASE-1697 ]
          Eugene Koontz made changes -
          Link This issue relates to ZOOKEEPER-938 [ ZOOKEEPER-938 ]
          Alex Newman made changes -
          Assignee Alex Newman [ posix4e ]
          Alex Newman made changes -
          Assignee Alex Newman [ posix4e ]
          Eugene Koontz made changes -
          Assignee Eugene Koontz [ ekoontz ]
          Eugene Koontz made changes -
          Link This issue is related to HBASE-3025 [ HBASE-3025 ]
          Andrew Purtell made changes -
          Link This issue blocks HBASE-3025 [ HBASE-3025 ]
          Andrew Purtell made changes -
          Link This issue is related to HBASE-3025 [ HBASE-3025 ]
          Eugene Koontz made changes -
          Labels security zookeeper
          Eugene Koontz made changes -
          Link This issue blocks HBASE-4791 [ HBASE-4791 ]
          stack made changes -
          Fix Version/s 0.92.0 [ 12314223 ]
          Andrew Purtell made changes -
          Attachment HBASE-2418-5.patch [ 12504322 ]
          Andrew Purtell made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hadoop Flags Reviewed [ 10343 ]
          Andrew Purtell made changes -
          Attachment HBASE-2418-5.patch [ 12504324 ]
          Andrew Purtell made changes -
          Attachment HBASE-2418-5.patch [ 12504329 ]
          Andrew Purtell made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Andrew Purtell made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Andrew Purtell made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Andrew Purtell made changes -
          Attachment HBASE-2418-6.patch [ 12504379 ]
          Andrew Purtell made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Andrew Purtell made changes -
          Attachment HBASE-2418-5.patch [ 12504329 ]
          Andrew Purtell made changes -
          Attachment HBASE-2418-5.patch [ 12504324 ]
          Andrew Purtell made changes -
          Attachment HBASE-2418-5.patch [ 12504322 ]
          Andrew Purtell made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Andrew Purtell made changes -
          Attachment HBASE-2418-6.patch [ 12504384 ]
          Andrew Purtell made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Andrew Purtell made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Andrew Purtell made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Andrew Purtell made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Release Note This adds support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

            Server {
              com.sun.security.auth.module.Krb5LoginModule required
              useKeyTab=true
              keyTab="/etc/hbase/conf/hbase.keytab"
              storeKey=true
              useTicketCache=false
              principal="zookeeper/$HOSTNAME";
            };
            Client {
              com.sun.security.auth.module.Krb5LoginModule required
              useKeyTab=true
              useTicketCache=false
              keyTab="/etc/hbase/conf/hbase.keytab"
              principal="hbase/$HOSTNAME";
            };

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

            HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
            HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"
            HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          We will pull in a Hadoop artifact patched with HADOOP-7070 if building under the security profile (-P security). 0.20.205 does not yet include HADOOP-7070. Without it, the JAAS configuration required for secure operation of the ZooKeeper client will be ignored.
          Fix Version/s 0.94.0 [ 12316419 ]
          Resolution Fixed [ 1 ]
          Ted Yu made changes -
          Attachment 2418.addendum [ 12504444 ]
          Eugene Koontz made changes -
          Link This issue relates to HBASE-4376 [ HBASE-4376 ]
          Eugene Koontz made changes -
          Link This issue blocks HBASE-4960 [ HBASE-4960 ]
          Jonathan Hsieh made changes -
          Link This issue blocks HBASE-5603 [ HBASE-5603 ]
          Lars Hofhansl made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Gavin made changes -
          Link This issue blocks HBASE-3025 [ HBASE-3025 ]
          Gavin made changes -
          Link This issue is depended upon by HBASE-3025 [ HBASE-3025 ]
          Gavin made changes -
          Link This issue blocks HBASE-4791 [ HBASE-4791 ]
          Gavin made changes -
          Link This issue is depended upon by HBASE-4791 [ HBASE-4791 ]
          Gavin made changes -
          Link This issue blocks HBASE-4960 [ HBASE-4960 ]
          Gavin made changes -
          Link This issue is depended upon by HBASE-4960 [ HBASE-4960 ]
          Gavin made changes -
          Link This issue blocks HBASE-5603 [ HBASE-5603 ]
          Gavin made changes -
          Link This issue is depended upon by HBASE-5603 [ HBASE-5603 ]

            People

            • Assignee:
              Eugene Koontz
              Reporter:
              Patrick Hunt
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development