HBase
  1. HBase
  2. HBASE-2418

add support for ZooKeeper authentication

    Details

    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      This adds support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

      SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

        Server {
          com.sun.security.auth.module.Krb5LoginModule required
          useKeyTab=true
          keyTab="/etc/hbase/conf/hbase.keytab"
          storeKey=true
          useTicketCache=false
          principal="zookeeper/$HOSTNAME";
        };
        Client {
          com.sun.security.auth.module.Krb5LoginModule required
          useKeyTab=true
          useTicketCache=false
          keyTab="/etc/hbase/conf/hbase.keytab"
          principal="hbase/$HOSTNAME";
        };

      and then configure both the client and server processes to use it, for example in hbase-site.xml:

        HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
        HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"
        HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

      HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

      We will pull in a Hadoop artifact patched with HADOOP-7070 if building under the security profile (-P security). 0.20.205 does not yet include HADOOP-7070. Without it, the JAAS configuration required for secure operation of the ZooKeeper client will be ignored.
      Show
      This adds support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:   Server {     com.sun.security.auth.module.Krb5LoginModule required     useKeyTab=true     keyTab="/etc/hbase/conf/hbase.keytab"     storeKey=true     useTicketCache=false     principal="zookeeper/$HOSTNAME";   };   Client {     com.sun.security.auth.module.Krb5LoginModule required     useKeyTab=true     useTicketCache=false     keyTab="/etc/hbase/conf/hbase.keytab"     principal="hbase/$HOSTNAME";   }; and then configure both the client and server processes to use it, for example in hbase-site.xml:   HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"   HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"   HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. We will pull in a Hadoop artifact patched with HADOOP-7070 if building under the security profile (-P security). 0.20.205 does not yet include HADOOP-7070 . Without it, the JAAS configuration required for secure operation of the ZooKeeper client will be ignored.

      Description

      Some users may run a ZooKeeper cluster in "multi tenant mode" meaning that more than one client service would
      like to share a single ZooKeeper service instance (cluster). In this case the client services typically want to protect
      their data (ZK znodes) from access by other services (tenants) on the cluster. Say you are running HBase and Solr
      and Neo4j, or multiple HBase instances, etc... having authentication/authorization on the znodes is important for both
      security and helping to ensure that services don't interact negatively (touch each other's data).

      Today HBase does not have support for authentication or authorization. This should be added to the HBase clients
      that are accessing the ZK cluster. In general it means calling addAuthInfo once after a session is established:

      http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooKeeper.html#addAuthInfo(java.lang.String, byte[])

      with a user specific credential, often times this is a shared secret or certificate. You may be able to statically configure this
      in some cases (config string or file to read from), however in my case in particular you may need to access it programmatically,
      which adds complexity as the end user may need to load code into HBase for accessing the credential.

      Secondly you need to specify a non "world" ACL when interacting with znodes (create primarily):
      http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/data/ACL.html
      http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/ZooDefs.html

      Feel free to ping the ZooKeeper team if you have questions. It might also be good to discuss with some
      potential end users - in particular regarding how the end user can specify the credential.

      1. HBASE-2418-6.patch
        25 kB
        Andrew Purtell
      2. HBASE-2418-6.patch
        25 kB
        Andrew Purtell
      3. 2418.addendum
        0.6 kB
        Ted Yu

        Issue Links

          Activity

          Hide
          Andrew Purtell added a comment -

          however in my case in particular you may need to access it programmatically, which adds complexity as the end user may need to load code into HBase for accessing the credential

          So this is some config on the HBase side involving reflection and HBase users would need to deploy some jar onto the HBase classpath, I assume.

          Show
          Andrew Purtell added a comment - however in my case in particular you may need to access it programmatically, which adds complexity as the end user may need to load code into HBase for accessing the credential So this is some config on the HBase side involving reflection and HBase users would need to deploy some jar onto the HBase classpath, I assume.
          Hide
          Alex Newman added a comment -

          Didn't mean to click that button

          Show
          Alex Newman added a comment - Didn't mean to click that button
          Hide
          Daniel Lescohier added a comment -

          The purpose of this issue is to prevent other (non-hbase) applications from accidentally writing to hbase's znodes on a multi-tenant zookeeper cluster [so this isn't really related to HBASE-1697 or HBASE-3025].

          I don't think we have to get too fancy; Zookeeper's Digest authentication would be sufficient for this use-case.

          Assuming that Hbase clients only need read-only access to hbase's zookeeper znodes, then this simple approache would be sufficient:

          Add a configuration property: hbase.zookeepeer.digest.auth.secrets.filename.

          The secret will go in a separate file so that you can deploy the secret only on the hbase servers; you don't want the secret directly in the hbase config file, since the hbase config file is also deployed to hbase clients.

          The contents of the secrets file will have one line with the data: username:password.

          The Hbase server code (master/regionserver) will change to:

          On server startup:
          -----------------

          if the property exists and the file exists and the file is readable:
          open the file and read the contents
          save the secret in server config state so that it can be used when connecting to zookeeper

          On connecting to zookeeper:
          --------------------------

          right after connected to zookeeper:
          if we have a digest secret from server startup:
          zk.addAuthInfo("digest", digest_secret);
          save in server config state:
          create_acl_list = ZooDefs.Ids.READ_ACL_UNSAFE.clone();
          create_acl_list.addAll(ZooDefs.Ids.CREATOR_ALL_ACL);
          // allows world: read access; hbase servers: all privs
          else: // don't use authentication
          save in server configuration state:
          create_acl_list = ZooDefs.Ids.OPEN_ACL_UNSAFE;

          On creating a node in zookeeper:
          -------------------------------
          pass create_acl_list as the acl parameter when calling Zookeeper.create()

          Show
          Daniel Lescohier added a comment - The purpose of this issue is to prevent other (non-hbase) applications from accidentally writing to hbase's znodes on a multi-tenant zookeeper cluster [so this isn't really related to HBASE-1697 or HBASE-3025] . I don't think we have to get too fancy; Zookeeper's Digest authentication would be sufficient for this use-case. Assuming that Hbase clients only need read-only access to hbase's zookeeper znodes, then this simple approache would be sufficient: Add a configuration property: hbase.zookeepeer.digest.auth.secrets.filename. The secret will go in a separate file so that you can deploy the secret only on the hbase servers; you don't want the secret directly in the hbase config file, since the hbase config file is also deployed to hbase clients. The contents of the secrets file will have one line with the data: username:password. The Hbase server code (master/regionserver) will change to: On server startup: ----------------- if the property exists and the file exists and the file is readable: open the file and read the contents save the secret in server config state so that it can be used when connecting to zookeeper On connecting to zookeeper: -------------------------- right after connected to zookeeper: if we have a digest secret from server startup: zk.addAuthInfo("digest", digest_secret); save in server config state: create_acl_list = ZooDefs.Ids.READ_ACL_UNSAFE.clone(); create_acl_list.addAll(ZooDefs.Ids.CREATOR_ALL_ACL); // allows world: read access; hbase servers: all privs else: // don't use authentication save in server configuration state: create_acl_list = ZooDefs.Ids.OPEN_ACL_UNSAFE; On creating a node in zookeeper: ------------------------------- pass create_acl_list as the acl parameter when calling Zookeeper.create()
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/
          -----------------------------------------------------------

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Summary
          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server

          { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/etc/hbase/conf/hbase.keytab" storeKey=true useTicketCache=false principal="zookeeper/$HOSTNAME"; }

          ;
          Client

          { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true useTicketCache=false keyTab="/etc/hbase/conf/hbase.keytab" principal="hbase/$HOSTNAME"; }

          ;

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="$

          {HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          HBASE_OPTS="${HBASE_OPTS}

          -Dzookeeper.kerberos.removeHostFromPrincipal=true"
          HBASE_OPTS="$

          {HBASE_OPTS}

          -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.
          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs


          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9
          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53
          pom.xml c74ce25
          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7

          Diff: https://reviews.apache.org/r/2837/diff

          Testing
          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- Review request for hbase, Gary Helmling and Eugene Koontz. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/etc/hbase/conf/hbase.keytab" storeKey=true useTicketCache=false principal="zookeeper/$HOSTNAME"; } ; Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true useTicketCache=false keyTab="/etc/hbase/conf/hbase.keytab" principal="hbase/$HOSTNAME"; } ; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="$ {HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="$ {HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 pom.xml c74ce25 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/#review3270
          -----------------------------------------------------------

          Patch looks good. There's a few questions below. You fellas are running this at TM?

          pom.xml
          <https://reviews.apache.org/r/2837/#comment7314>

          What are the implications of shipping a 3.4 zk snapshot with 0.92 hbase? Will a 3.4 client be able to talk to a 3.3.3. ensemble?

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          <https://reviews.apache.org/r/2837/#comment7316>

          So, how do we guarantee that our session w/ zk is secure if the hbase install is configured secure? What is in place to prevent our connecting insecure to a zk ensemble if the hbase is supposed to be secure?

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          <https://reviews.apache.org/r/2837/#comment7315>

          Would suggest filing the jira and reference it here.

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
          <https://reviews.apache.org/r/2837/#comment7317>

          Good

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java
          <https://reviews.apache.org/r/2837/#comment7318>

          This line is no longer needed.

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java
          <https://reviews.apache.org/r/2837/#comment7319>

          This is an insecure cluster using a secure zk?

          • Michael

          On 2011-11-15 19:43:37, Andrew Purtell wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2837/

          -----------------------------------------------------------

          (Updated 2011-11-15 19:43:37)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Summary

          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. };

          Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. };

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"

          HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.

          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs

          -----

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          pom.xml c74ce25

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7

          Diff: https://reviews.apache.org/r/2837/diff

          Testing

          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/#review3270 ----------------------------------------------------------- Patch looks good. There's a few questions below. You fellas are running this at TM? pom.xml < https://reviews.apache.org/r/2837/#comment7314 > What are the implications of shipping a 3.4 zk snapshot with 0.92 hbase? Will a 3.4 client be able to talk to a 3.3.3. ensemble? src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java < https://reviews.apache.org/r/2837/#comment7316 > So, how do we guarantee that our session w/ zk is secure if the hbase install is configured secure? What is in place to prevent our connecting insecure to a zk ensemble if the hbase is supposed to be secure? src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java < https://reviews.apache.org/r/2837/#comment7315 > Would suggest filing the jira and reference it here. src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java < https://reviews.apache.org/r/2837/#comment7317 > Good src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java < https://reviews.apache.org/r/2837/#comment7318 > This line is no longer needed. src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java < https://reviews.apache.org/r/2837/#comment7319 > This is an insecure cluster using a secure zk? Michael On 2011-11-15 19:43:37, Andrew Purtell wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-15 19:43:37) Review request for hbase, Gary Helmling and Eugene Koontz. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. }; Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. }; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs ----- src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 pom.xml c74ce25 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/#review3272
          -----------------------------------------------------------

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          <https://reviews.apache.org/r/2837/#comment7320>

          I agree that simply checking for a configuration file is insufficient.

          We should use the JAAS configuration class (http://download.oracle.com/javase/1.4.2/docs/api/javax/security/auth/login/Configuration.html) API to parse the supplied JAAS configuration and make sure that the "Client" section exists. We could also log some warnings or info based on what we find there (for example, if the "Client" section exists, but there's no principal, LOG.warn("No principal found in Client section").

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
          <https://reviews.apache.org/r/2837/#comment7321>

          s/Its/It's/

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
          <https://reviews.apache.org/r/2837/#comment7322>

          Note that this fix is also in the patch for HBASE-3861: https://issues.apache.org/jira/secure/attachment/12478359/HBASE-3861.patch

          • Eugene

          On 2011-11-15 19:43:37, Andrew Purtell wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2837/

          -----------------------------------------------------------

          (Updated 2011-11-15 19:43:37)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Summary

          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. };

          Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. };

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"

          HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.

          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs

          -----

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          pom.xml c74ce25

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7

          Diff: https://reviews.apache.org/r/2837/diff

          Testing

          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/#review3272 ----------------------------------------------------------- src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java < https://reviews.apache.org/r/2837/#comment7320 > I agree that simply checking for a configuration file is insufficient. We should use the JAAS configuration class ( http://download.oracle.com/javase/1.4.2/docs/api/javax/security/auth/login/Configuration.html ) API to parse the supplied JAAS configuration and make sure that the "Client" section exists. We could also log some warnings or info based on what we find there (for example, if the "Client" section exists, but there's no principal, LOG.warn("No principal found in Client section"). src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java < https://reviews.apache.org/r/2837/#comment7321 > s/Its/It's/ src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java < https://reviews.apache.org/r/2837/#comment7322 > Note that this fix is also in the patch for HBASE-3861 : https://issues.apache.org/jira/secure/attachment/12478359/HBASE-3861.patch Eugene On 2011-11-15 19:43:37, Andrew Purtell wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-15 19:43:37) Review request for hbase, Gary Helmling and Eugene Koontz. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. }; Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. }; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs ----- src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 pom.xml c74ce25 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/#review3277
          -----------------------------------------------------------

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java
          <https://reviews.apache.org/r/2837/#comment7334>

          Maybe add something about final test, since we are summarizing the other tests:

          "Finally, we check the ACLs of a node outside of the /hbase hierarchy and verify that its ACL is simply 'hbase:Perms.ALL'."

          • Eugene

          On 2011-11-15 19:43:37, Andrew Purtell wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2837/

          -----------------------------------------------------------

          (Updated 2011-11-15 19:43:37)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Summary

          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. };

          Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. };

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"

          HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.

          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs

          -----

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          pom.xml c74ce25

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7

          Diff: https://reviews.apache.org/r/2837/diff

          Testing

          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/#review3277 ----------------------------------------------------------- src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java < https://reviews.apache.org/r/2837/#comment7334 > Maybe add something about final test, since we are summarizing the other tests: "Finally, we check the ACLs of a node outside of the /hbase hierarchy and verify that its ACL is simply 'hbase:Perms.ALL'." Eugene On 2011-11-15 19:43:37, Andrew Purtell wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-15 19:43:37) Review request for hbase, Gary Helmling and Eugene Koontz. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. }; Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. }; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs ----- src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 pom.xml c74ce25 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-15 20:12:11, Michael Stack wrote:

          > pom.xml, line 794

          > <https://reviews.apache.org/r/2837/diff/1/?file=58410#file58410line794>

          >

          > What are the implications of shipping a 3.4 zk snapshot with 0.92 hbase? Will a 3.4 client be able to talk to a 3.3.3. ensemble?

          Will a 3.4 client be able to talk to a 3.3.3. ensemble?

          No, but we can make this change conditional on enabling the Maven '-P security' profile. I should have done that already.

          On 2011-11-15 20:12:11, Michael Stack wrote:

          > src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java, line 668

          > <https://reviews.apache.org/r/2837/diff/1/?file=58412#file58412line668>

          >

          > So, how do we guarantee that our session w/ zk is secure if the hbase install is configured secure? What is in place to prevent our connecting insecure to a zk ensemble if the hbase is supposed to be secure?

          ZooKeeper auth is independent of RPC auth.

          Connecting insecurely to a secure ZK ensemble is totally acceptable from a non-HBase point of view. It allows backwards compatibility over in ZK land.

          Even when running secure HBase, most clients would have no trouble even if connecting insecurely to ZK; only the Master and RegionServers would want to authenticate and set ACLs accordingly.

          We could add another check elsewhere in the Master and RegionServer (via ZKUtil presumably) if HBase security is enabled to test that ACLs are set up, but this won't let someone run with an insecure ZooKeeper version, maybe 3.3.3 or whatever. Maybe someone will want that. I think it's a user concern.

          On 2011-11-15 20:12:11, Michael Stack wrote:

          > src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java, line 676

          > <https://reviews.apache.org/r/2837/diff/1/?file=58412#file58412line676>

          >

          > Would suggest filing the jira and reference it here.

          Eugene opened HBASE-4791. Will make a note.

          On 2011-11-15 20:12:11, Michael Stack wrote:

          > src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java, line 77

          > <https://reviews.apache.org/r/2837/diff/1/?file=58415#file58415line77>

          >

          > This is an insecure cluster using a secure zk?

          ZooKeeper auth is independent of RPC auth.

          • Andrew

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/#review3270
          -----------------------------------------------------------

          On 2011-11-15 19:43:37, Andrew Purtell wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2837/

          -----------------------------------------------------------

          (Updated 2011-11-15 19:43:37)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Summary

          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. };

          Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. };

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"

          HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.

          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs

          -----

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          pom.xml c74ce25

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7

          Diff: https://reviews.apache.org/r/2837/diff

          Testing

          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-15 20:12:11, Michael Stack wrote: > pom.xml, line 794 > < https://reviews.apache.org/r/2837/diff/1/?file=58410#file58410line794 > > > What are the implications of shipping a 3.4 zk snapshot with 0.92 hbase? Will a 3.4 client be able to talk to a 3.3.3. ensemble? Will a 3.4 client be able to talk to a 3.3.3. ensemble? No, but we can make this change conditional on enabling the Maven '-P security' profile. I should have done that already. On 2011-11-15 20:12:11, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java, line 668 > < https://reviews.apache.org/r/2837/diff/1/?file=58412#file58412line668 > > > So, how do we guarantee that our session w/ zk is secure if the hbase install is configured secure? What is in place to prevent our connecting insecure to a zk ensemble if the hbase is supposed to be secure? ZooKeeper auth is independent of RPC auth. Connecting insecurely to a secure ZK ensemble is totally acceptable from a non-HBase point of view. It allows backwards compatibility over in ZK land. Even when running secure HBase, most clients would have no trouble even if connecting insecurely to ZK; only the Master and RegionServers would want to authenticate and set ACLs accordingly. We could add another check elsewhere in the Master and RegionServer (via ZKUtil presumably) if HBase security is enabled to test that ACLs are set up, but this won't let someone run with an insecure ZooKeeper version, maybe 3.3.3 or whatever. Maybe someone will want that. I think it's a user concern. On 2011-11-15 20:12:11, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java, line 676 > < https://reviews.apache.org/r/2837/diff/1/?file=58412#file58412line676 > > > Would suggest filing the jira and reference it here. Eugene opened HBASE-4791 . Will make a note. On 2011-11-15 20:12:11, Michael Stack wrote: > src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java, line 77 > < https://reviews.apache.org/r/2837/diff/1/?file=58415#file58415line77 > > > This is an insecure cluster using a secure zk? ZooKeeper auth is independent of RPC auth. Andrew ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/#review3270 ----------------------------------------------------------- On 2011-11-15 19:43:37, Andrew Purtell wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-15 19:43:37) Review request for hbase, Gary Helmling and Eugene Koontz. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. }; Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. }; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs ----- src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 pom.xml c74ce25 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-15 20:15:21, Eugene Koontz wrote:

          > src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java, line 669

          > <https://reviews.apache.org/r/2837/diff/1/?file=58412#file58412line669>

          >

          > I agree that simply checking for a configuration file is insufficient.

          >

          > We should use the JAAS configuration class (http://download.oracle.com/javase/1.4.2/docs/api/javax/security/auth/login/Configuration.html) API to parse the supplied JAAS configuration and make sure that the "Client" section exists. We could also log some warnings or info based on what we find there (for example, if the "Client" section exists, but there's no principal, LOG.warn("No principal found in Client section").

          Thanks for opening HBASE-4791. I think we can follow up on this there.

          • Andrew

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/#review3272
          -----------------------------------------------------------

          On 2011-11-15 19:43:37, Andrew Purtell wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2837/

          -----------------------------------------------------------

          (Updated 2011-11-15 19:43:37)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Summary

          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. };

          Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. };

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"

          HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.

          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs

          -----

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          pom.xml c74ce25

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7

          Diff: https://reviews.apache.org/r/2837/diff

          Testing

          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-15 20:15:21, Eugene Koontz wrote: > src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java, line 669 > < https://reviews.apache.org/r/2837/diff/1/?file=58412#file58412line669 > > > I agree that simply checking for a configuration file is insufficient. > > We should use the JAAS configuration class ( http://download.oracle.com/javase/1.4.2/docs/api/javax/security/auth/login/Configuration.html ) API to parse the supplied JAAS configuration and make sure that the "Client" section exists. We could also log some warnings or info based on what we find there (for example, if the "Client" section exists, but there's no principal, LOG.warn("No principal found in Client section"). Thanks for opening HBASE-4791 . I think we can follow up on this there. Andrew ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/#review3272 ----------------------------------------------------------- On 2011-11-15 19:43:37, Andrew Purtell wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-15 19:43:37) Review request for hbase, Gary Helmling and Eugene Koontz. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. }; Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. }; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs ----- src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 pom.xml c74ce25 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-15 20:12:11, Michael Stack wrote:

          > src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java, line 676

          > <https://reviews.apache.org/r/2837/diff/1/?file=58412#file58412line676>

          >

          > Would suggest filing the jira and reference it here.

          Andrew Purtell wrote:

          Eugene opened HBASE-4791. Will make a note.

          JIRA created and linked to HBASE-2418: https://issues.apache.org/jira/browse/HBASE-4791

          On 2011-11-15 20:12:11, Michael Stack wrote:

          > src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java, line 77

          > <https://reviews.apache.org/r/2837/diff/1/?file=58415#file58415line77>

          >

          > This is an insecure cluster using a secure zk?

          Andrew Purtell wrote:

          ZooKeeper auth is independent of RPC auth.

          Both zookeeper server and client are using SASL authentication since there are both "Server" and "Client" sections in the jaas.conf file. So HBase and Zookeeper authenticate with each other in this test.

          • Eugene

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/#review3270
          -----------------------------------------------------------

          On 2011-11-15 19:43:37, Andrew Purtell wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2837/

          -----------------------------------------------------------

          (Updated 2011-11-15 19:43:37)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Summary

          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. };

          Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. };

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"

          HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.

          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs

          -----

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          pom.xml c74ce25

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7

          Diff: https://reviews.apache.org/r/2837/diff

          Testing

          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-15 20:12:11, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java, line 676 > < https://reviews.apache.org/r/2837/diff/1/?file=58412#file58412line676 > > > Would suggest filing the jira and reference it here. Andrew Purtell wrote: Eugene opened HBASE-4791 . Will make a note. JIRA created and linked to HBASE-2418 : https://issues.apache.org/jira/browse/HBASE-4791 On 2011-11-15 20:12:11, Michael Stack wrote: > src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java, line 77 > < https://reviews.apache.org/r/2837/diff/1/?file=58415#file58415line77 > > > This is an insecure cluster using a secure zk? Andrew Purtell wrote: ZooKeeper auth is independent of RPC auth. Both zookeeper server and client are using SASL authentication since there are both "Server" and "Client" sections in the jaas.conf file. So HBase and Zookeeper authenticate with each other in this test. Eugene ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/#review3270 ----------------------------------------------------------- On 2011-11-15 19:43:37, Andrew Purtell wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-15 19:43:37) Review request for hbase, Gary Helmling and Eugene Koontz. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. }; Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. }; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs ----- src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 pom.xml c74ce25 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/
          -----------------------------------------------------------

          (Updated 2011-11-15 23:15:34.845036)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Changes
          -------

          Updated patch addresses review comments to date that have specific suggestions.

          Summary
          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server

          { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/etc/hbase/conf/hbase.keytab" storeKey=true useTicketCache=false principal="zookeeper/$HOSTNAME"; }

          ;
          Client

          { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true useTicketCache=false keyTab="/etc/hbase/conf/hbase.keytab" principal="hbase/$HOSTNAME"; }

          ;

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="$

          {HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          HBASE_OPTS="${HBASE_OPTS}

          -Dzookeeper.kerberos.removeHostFromPrincipal=true"
          HBASE_OPTS="$

          {HBASE_OPTS}

          -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.
          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs (updated)


          pom.xml c74ce25
          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87
          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9
          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2837/diff

          Testing
          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-15 23:15:34.845036) Review request for hbase, Gary Helmling and Eugene Koontz. Changes ------- Updated patch addresses review comments to date that have specific suggestions. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/etc/hbase/conf/hbase.keytab" storeKey=true useTicketCache=false principal="zookeeper/$HOSTNAME"; } ; Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true useTicketCache=false keyTab="/etc/hbase/conf/hbase.keytab" principal="hbase/$HOSTNAME"; } ; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="$ {HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="$ {HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs (updated) pom.xml c74ce25 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/#review3286
          -----------------------------------------------------------

          pom.xml
          <https://reviews.apache.org/r/2837/#comment7352>

          This addresses the comment about shipping a 3.4-SNAPSHOT if it is not warranted. The POM stanza here is a dup from related changes in HBASE-3025 with the zookeeper.version property added.

          • Andrew

          On 2011-11-15 23:15:34, Andrew Purtell wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2837/

          -----------------------------------------------------------

          (Updated 2011-11-15 23:15:34)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Summary

          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. };

          Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. };

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"

          HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.

          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs

          -----

          pom.xml c74ce25

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2837/diff

          Testing

          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/#review3286 ----------------------------------------------------------- pom.xml < https://reviews.apache.org/r/2837/#comment7352 > This addresses the comment about shipping a 3.4-SNAPSHOT if it is not warranted. The POM stanza here is a dup from related changes in HBASE-3025 with the zookeeper.version property added. Andrew On 2011-11-15 23:15:34, Andrew Purtell wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-15 23:15:34) Review request for hbase, Gary Helmling and Eugene Koontz. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. }; Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. }; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs ----- pom.xml c74ce25 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/
          -----------------------------------------------------------

          (Updated 2011-11-17 20:58:47.295983)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Changes
          -------

          Updated POM to pull in Hadoop 0.20.205.1-7070-SNAPSHOT if building under the security profile (-P security). 0.20.205 does not yet include HADOOP-7070. Without it, the JAAS configuration required for secure operation of the ZooKeeper client will be ignored. This is not an issue with this patch per se. Hadoop without 7070 overrides any JAAS configuration. ZooKeeper integrated security in such a way as to require one. HBase is stuck in the middle here.

          We will file another JIRA for simplifying the configuration of a secure HBase cluster. The user should need only to update one or two configuration properties in hbase-site.xml with the remainder handled behind the scenes. Within the context of that JIRA we can look at having HBase build the ZooKeeper JAAS configuration programatically, like Hadoop does. We may be able to simply update the LoginContext that Hadoop provides.

          Summary
          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server

          { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/etc/hbase/conf/hbase.keytab" storeKey=true useTicketCache=false principal="zookeeper/$HOSTNAME"; }

          ;
          Client

          { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true useTicketCache=false keyTab="/etc/hbase/conf/hbase.keytab" principal="hbase/$HOSTNAME"; }

          ;

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="$

          {HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          HBASE_OPTS="${HBASE_OPTS}

          -Dzookeeper.kerberos.removeHostFromPrincipal=true"
          HBASE_OPTS="$

          {HBASE_OPTS}

          -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.
          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs (updated)


          pom.xml c74ce25
          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87
          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9
          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2837/diff

          Testing
          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-17 20:58:47.295983) Review request for hbase, Gary Helmling and Eugene Koontz. Changes ------- Updated POM to pull in Hadoop 0.20.205.1-7070-SNAPSHOT if building under the security profile (-P security). 0.20.205 does not yet include HADOOP-7070 . Without it, the JAAS configuration required for secure operation of the ZooKeeper client will be ignored. This is not an issue with this patch per se. Hadoop without 7070 overrides any JAAS configuration. ZooKeeper integrated security in such a way as to require one. HBase is stuck in the middle here. We will file another JIRA for simplifying the configuration of a secure HBase cluster. The user should need only to update one or two configuration properties in hbase-site.xml with the remainder handled behind the scenes. Within the context of that JIRA we can look at having HBase build the ZooKeeper JAAS configuration programatically, like Hadoop does. We may be able to simply update the LoginContext that Hadoop provides. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/etc/hbase/conf/hbase.keytab" storeKey=true useTicketCache=false principal="zookeeper/$HOSTNAME"; } ; Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true useTicketCache=false keyTab="/etc/hbase/conf/hbase.keytab" principal="hbase/$HOSTNAME"; } ; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="$ {HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="$ {HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs (updated) pom.xml c74ce25 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/#review3335
          -----------------------------------------------------------

          Ship it!

          +1 on commit (if all tests pass and my questions below have the answers i want)

          pom.xml
          <https://reviews.apache.org/r/2837/#comment7446>

          This stuff is also in other patches? I suppose we'll see who goes in first.

          pom.xml
          <https://reviews.apache.org/r/2837/#comment7449>

          This is in gary's repo? Is this the only version we'll work with?

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
          <https://reviews.apache.org/r/2837/#comment7450>

          These are not zk 3.4 imports are they?

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
          <https://reviews.apache.org/r/2837/#comment7451>

          Is this a 3.4 zk change?

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
          <https://reviews.apache.org/r/2837/#comment7453>

          Are these only in zk 3.4?

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java
          <https://reviews.apache.org/r/2837/#comment7455>

          This will work w/ 3.3.3. zk?

          • Michael

          On 2011-11-17 20:58:47, Andrew Purtell wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2837/

          -----------------------------------------------------------

          (Updated 2011-11-17 20:58:47)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Summary

          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. };

          Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. };

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"

          HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.

          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs

          -----

          pom.xml c74ce25

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2837/diff

          Testing

          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/#review3335 ----------------------------------------------------------- Ship it! +1 on commit (if all tests pass and my questions below have the answers i want) pom.xml < https://reviews.apache.org/r/2837/#comment7446 > This stuff is also in other patches? I suppose we'll see who goes in first. pom.xml < https://reviews.apache.org/r/2837/#comment7449 > This is in gary's repo? Is this the only version we'll work with? src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java < https://reviews.apache.org/r/2837/#comment7450 > These are not zk 3.4 imports are they? src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java < https://reviews.apache.org/r/2837/#comment7451 > Is this a 3.4 zk change? src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java < https://reviews.apache.org/r/2837/#comment7453 > Are these only in zk 3.4? src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java < https://reviews.apache.org/r/2837/#comment7455 > This will work w/ 3.3.3. zk? Michael On 2011-11-17 20:58:47, Andrew Purtell wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-17 20:58:47) Review request for hbase, Gary Helmling and Eugene Koontz. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. }; Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. }; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs ----- pom.xml c74ce25 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-17 23:21:44, Michael Stack wrote:

          > +1 on commit (if all tests pass and my questions below have the answers i want)

          -1 based on review, thanks.

          On 2011-11-17 23:21:44, Michael Stack wrote:

          > pom.xml, line 251

          > <https://reviews.apache.org/r/2837/diff/3/?file=59198#file59198line251>

          >

          > This stuff is also in other patches? I suppose we'll see who goes in first.

          Yes

          On 2011-11-17 23:21:44, Michael Stack wrote:

          > src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java, line 39

          > <https://reviews.apache.org/r/2837/diff/3/?file=59199#file59199line39>

          >

          > These are not zk 3.4 imports are they?

          You are right, this is not going to work without some kind of shim.

          On 2011-11-17 23:21:44, Michael Stack wrote:

          > pom.xml, line 1354

          > <https://reviews.apache.org/r/2837/diff/3/?file=59198#file59198line1354>

          >

          > This is in gary's repo? Is this the only version we'll work with?

          Without 7070 Hadoop will clobber any JAAS configuration supplied for ZK. So a Hadoop with 7070 must be used if one wants to set up ZK 3.4 for SASL authentication.

          • Andrew

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/#review3335
          -----------------------------------------------------------

          On 2011-11-17 20:58:47, Andrew Purtell wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2837/

          -----------------------------------------------------------

          (Updated 2011-11-17 20:58:47)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Summary

          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. };

          Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. };

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"

          HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.

          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs

          -----

          pom.xml c74ce25

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2837/diff

          Testing

          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-17 23:21:44, Michael Stack wrote: > +1 on commit (if all tests pass and my questions below have the answers i want) -1 based on review, thanks. On 2011-11-17 23:21:44, Michael Stack wrote: > pom.xml, line 251 > < https://reviews.apache.org/r/2837/diff/3/?file=59198#file59198line251 > > > This stuff is also in other patches? I suppose we'll see who goes in first. Yes On 2011-11-17 23:21:44, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java, line 39 > < https://reviews.apache.org/r/2837/diff/3/?file=59199#file59199line39 > > > These are not zk 3.4 imports are they? You are right, this is not going to work without some kind of shim. On 2011-11-17 23:21:44, Michael Stack wrote: > pom.xml, line 1354 > < https://reviews.apache.org/r/2837/diff/3/?file=59198#file59198line1354 > > > This is in gary's repo? Is this the only version we'll work with? Without 7070 Hadoop will clobber any JAAS configuration supplied for ZK. So a Hadoop with 7070 must be used if one wants to set up ZK 3.4 for SASL authentication. Andrew ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/#review3335 ----------------------------------------------------------- On 2011-11-17 20:58:47, Andrew Purtell wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-17 20:58:47) Review request for hbase, Gary Helmling and Eugene Koontz. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. }; Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. }; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs ----- pom.xml c74ce25 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/#review3344
          -----------------------------------------------------------

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
          <https://reviews.apache.org/r/2837/#comment7486>

          I don't see how to work around this. The code has to be recompiled against 3.3 or 3.4.

          Any ideas?

          I think we have to pull in 3.4 unconditionally.

          • Andrew

          On 2011-11-17 20:58:47, Andrew Purtell wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2837/

          -----------------------------------------------------------

          (Updated 2011-11-17 20:58:47)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Summary

          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. };

          Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. };

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"

          HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.

          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs

          -----

          pom.xml c74ce25

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2837/diff

          Testing

          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/#review3344 ----------------------------------------------------------- src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java < https://reviews.apache.org/r/2837/#comment7486 > I don't see how to work around this. The code has to be recompiled against 3.3 or 3.4. Any ideas? I think we have to pull in 3.4 unconditionally. Andrew On 2011-11-17 20:58:47, Andrew Purtell wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-17 20:58:47) Review request for hbase, Gary Helmling and Eugene Koontz. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. }; Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. }; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs ----- pom.xml c74ce25 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-18 04:14:48, Andrew Purtell wrote:

          > src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java, line 338

          > <https://reviews.apache.org/r/2837/diff/3/?file=59201#file59201line338>

          >

          > I don't see how to work around this. The code has to be recompiled against 3.3 or 3.4.

          >

          > Any ideas?

          >

          > I think we have to pull in 3.4 unconditionally.

          Might be a good thing anyway. 3.4 is in RC right now, would probably be GA by the time we GA 0.92. (we're meeting with Ted Dunning tomorrow - unrelated - can ask him what he thinks about it).

          • Lars

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/#review3344
          -----------------------------------------------------------

          On 2011-11-17 20:58:47, Andrew Purtell wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2837/

          -----------------------------------------------------------

          (Updated 2011-11-17 20:58:47)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Summary

          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. };

          Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. };

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"

          HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.

          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs

          -----

          pom.xml c74ce25

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2837/diff

          Testing

          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-18 04:14:48, Andrew Purtell wrote: > src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java, line 338 > < https://reviews.apache.org/r/2837/diff/3/?file=59201#file59201line338 > > > I don't see how to work around this. The code has to be recompiled against 3.3 or 3.4. > > Any ideas? > > I think we have to pull in 3.4 unconditionally. Might be a good thing anyway. 3.4 is in RC right now, would probably be GA by the time we GA 0.92. (we're meeting with Ted Dunning tomorrow - unrelated - can ask him what he thinks about it). Lars ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/#review3344 ----------------------------------------------------------- On 2011-11-17 20:58:47, Andrew Purtell wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-17 20:58:47) Review request for hbase, Gary Helmling and Eugene Koontz. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. }; Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. }; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs ----- pom.xml c74ce25 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-18 04:14:48, Andrew Purtell wrote:

          > src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java, line 338

          > <https://reviews.apache.org/r/2837/diff/3/?file=59201#file59201line338>

          >

          > I don't see how to work around this. The code has to be recompiled against 3.3 or 3.4.

          >

          > Any ideas?

          >

          > I think we have to pull in 3.4 unconditionally.

          Lars Hofhansl wrote:

          Might be a good thing anyway. 3.4 is in RC right now, would probably be GA by the time we GA 0.92. (we're meeting with Ted Dunning tomorrow - unrelated - can ask him what he thinks about it).

          We can put in the other two patches and defer this one until 3.4 is released. Consequence would be that 3025 can be subverted if one allows direct client access to the ZK cluster.

          • Andrew

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/#review3344
          -----------------------------------------------------------

          On 2011-11-17 20:58:47, Andrew Purtell wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2837/

          -----------------------------------------------------------

          (Updated 2011-11-17 20:58:47)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Summary

          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. };

          Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. };

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"

          HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.

          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs

          -----

          pom.xml c74ce25

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2837/diff

          Testing

          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-18 04:14:48, Andrew Purtell wrote: > src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java, line 338 > < https://reviews.apache.org/r/2837/diff/3/?file=59201#file59201line338 > > > I don't see how to work around this. The code has to be recompiled against 3.3 or 3.4. > > Any ideas? > > I think we have to pull in 3.4 unconditionally. Lars Hofhansl wrote: Might be a good thing anyway. 3.4 is in RC right now, would probably be GA by the time we GA 0.92. (we're meeting with Ted Dunning tomorrow - unrelated - can ask him what he thinks about it). We can put in the other two patches and defer this one until 3.4 is released. Consequence would be that 3025 can be subverted if one allows direct client access to the ZK cluster. Andrew ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/#review3344 ----------------------------------------------------------- On 2011-11-17 20:58:47, Andrew Purtell wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-17 20:58:47) Review request for hbase, Gary Helmling and Eugene Koontz. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. }; Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. }; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs ----- pom.xml c74ce25 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-18 04:14:48, Andrew Purtell wrote:

          > src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java, line 338

          > <https://reviews.apache.org/r/2837/diff/3/?file=59201#file59201line338>

          >

          > I don't see how to work around this. The code has to be recompiled against 3.3 or 3.4.

          >

          > Any ideas?

          >

          > I think we have to pull in 3.4 unconditionally.

          Lars Hofhansl wrote:

          Might be a good thing anyway. 3.4 is in RC right now, would probably be GA by the time we GA 0.92. (we're meeting with Ted Dunning tomorrow - unrelated - can ask him what he thinks about it).

          Andrew Purtell wrote:

          We can put in the other two patches and defer this one until 3.4 is released. Consequence would be that 3025 can be subverted if one allows direct client access to the ZK cluster.

          +1 on fixing this patch so it just upgrades our zk to 3.4. I just tried hbase w/ a 3.4 client and a 3.3.3. ensemble and it seems to work. I asked Mahadev to be sure and he says "A 3.3.* client should be able to talk to 3.4.0 server and vice versa." Its as yet unreleased but I'm pretty sure it'll be out before we ship – maybe even a 3.4.1 (smile).

          • Michael

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/#review3344
          -----------------------------------------------------------

          On 2011-11-17 20:58:47, Andrew Purtell wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2837/

          -----------------------------------------------------------

          (Updated 2011-11-17 20:58:47)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Summary

          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. };

          Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. };

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"

          HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.

          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs

          -----

          pom.xml c74ce25

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2837/diff

          Testing

          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-18 04:14:48, Andrew Purtell wrote: > src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java, line 338 > < https://reviews.apache.org/r/2837/diff/3/?file=59201#file59201line338 > > > I don't see how to work around this. The code has to be recompiled against 3.3 or 3.4. > > Any ideas? > > I think we have to pull in 3.4 unconditionally. Lars Hofhansl wrote: Might be a good thing anyway. 3.4 is in RC right now, would probably be GA by the time we GA 0.92. (we're meeting with Ted Dunning tomorrow - unrelated - can ask him what he thinks about it). Andrew Purtell wrote: We can put in the other two patches and defer this one until 3.4 is released. Consequence would be that 3025 can be subverted if one allows direct client access to the ZK cluster. +1 on fixing this patch so it just upgrades our zk to 3.4. I just tried hbase w/ a 3.4 client and a 3.3.3. ensemble and it seems to work. I asked Mahadev to be sure and he says "A 3.3.* client should be able to talk to 3.4.0 server and vice versa." Its as yet unreleased but I'm pretty sure it'll be out before we ship – maybe even a 3.4.1 (smile). Michael ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/#review3344 ----------------------------------------------------------- On 2011-11-17 20:58:47, Andrew Purtell wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-17 20:58:47) Review request for hbase, Gary Helmling and Eugene Koontz. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. }; Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. }; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs ----- pom.xml c74ce25 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java bb67e53 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/
          -----------------------------------------------------------

          (Updated 2011-11-18 16:05:49.921654)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Changes
          -------

          Unconditionally pull in ZK 3.4.

          Some of the POM changes might reject, depending on what goes in first, but would be trivial to fix up.

          Summary
          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server

          { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/etc/hbase/conf/hbase.keytab" storeKey=true useTicketCache=false principal="zookeeper/$HOSTNAME"; }

          ;
          Client

          { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true useTicketCache=false keyTab="/etc/hbase/conf/hbase.keytab" principal="hbase/$HOSTNAME"; }

          ;

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="$

          {HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          HBASE_OPTS="${HBASE_OPTS}

          -Dzookeeper.kerberos.removeHostFromPrincipal=true"
          HBASE_OPTS="$

          {HBASE_OPTS}

          -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.
          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs (updated)


          pom.xml 382c7c2
          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 74b9e62
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87
          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9
          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2837/diff

          Testing
          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-18 16:05:49.921654) Review request for hbase, Gary Helmling and Eugene Koontz. Changes ------- Unconditionally pull in ZK 3.4. Some of the POM changes might reject, depending on what goes in first, but would be trivial to fix up. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/etc/hbase/conf/hbase.keytab" storeKey=true useTicketCache=false principal="zookeeper/$HOSTNAME"; } ; Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true useTicketCache=false keyTab="/etc/hbase/conf/hbase.keytab" principal="hbase/$HOSTNAME"; } ; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="$ {HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="$ {HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs (updated) pom.xml 382c7c2 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 74b9e62 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-18 04:14:48, Andrew Purtell wrote:

          > src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java, line 338

          > <https://reviews.apache.org/r/2837/diff/3/?file=59201#file59201line338>

          >

          > I don't see how to work around this. The code has to be recompiled against 3.3 or 3.4.

          >

          > Any ideas?

          >

          > I think we have to pull in 3.4 unconditionally.

          Lars Hofhansl wrote:

          Might be a good thing anyway. 3.4 is in RC right now, would probably be GA by the time we GA 0.92. (we're meeting with Ted Dunning tomorrow - unrelated - can ask him what he thinks about it).

          Andrew Purtell wrote:

          We can put in the other two patches and defer this one until 3.4 is released. Consequence would be that 3025 can be subverted if one allows direct client access to the ZK cluster.

          Michael Stack wrote:

          +1 on fixing this patch so it just upgrades our zk to 3.4. I just tried hbase w/ a 3.4 client and a 3.3.3. ensemble and it seems to work. I asked Mahadev to be sure and he says "A 3.3.* client should be able to talk to 3.4.0 server and vice versa." Its as yet unreleased but I'm pretty sure it'll be out before we ship – maybe even a 3.4.1 (smile).

          Say what you will about the Hadoop Record compiler aka Jute, but ZK is 1) the only project that uses it that I know of 2) yet consistently can answer in the affirmative regarding cross-version compatibility.

          • Andrew

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/#review3344
          -----------------------------------------------------------

          On 2011-11-18 16:05:49, Andrew Purtell wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2837/

          -----------------------------------------------------------

          (Updated 2011-11-18 16:05:49)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Summary

          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. };

          Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. };

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"

          HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.

          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs

          -----

          pom.xml 382c7c2

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 74b9e62

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2837/diff

          Testing

          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-18 04:14:48, Andrew Purtell wrote: > src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java, line 338 > < https://reviews.apache.org/r/2837/diff/3/?file=59201#file59201line338 > > > I don't see how to work around this. The code has to be recompiled against 3.3 or 3.4. > > Any ideas? > > I think we have to pull in 3.4 unconditionally. Lars Hofhansl wrote: Might be a good thing anyway. 3.4 is in RC right now, would probably be GA by the time we GA 0.92. (we're meeting with Ted Dunning tomorrow - unrelated - can ask him what he thinks about it). Andrew Purtell wrote: We can put in the other two patches and defer this one until 3.4 is released. Consequence would be that 3025 can be subverted if one allows direct client access to the ZK cluster. Michael Stack wrote: +1 on fixing this patch so it just upgrades our zk to 3.4. I just tried hbase w/ a 3.4 client and a 3.3.3. ensemble and it seems to work. I asked Mahadev to be sure and he says "A 3.3.* client should be able to talk to 3.4.0 server and vice versa." Its as yet unreleased but I'm pretty sure it'll be out before we ship – maybe even a 3.4.1 (smile). Say what you will about the Hadoop Record compiler aka Jute, but ZK is 1) the only project that uses it that I know of 2) yet consistently can answer in the affirmative regarding cross-version compatibility. Andrew ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/#review3344 ----------------------------------------------------------- On 2011-11-18 16:05:49, Andrew Purtell wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-18 16:05:49) Review request for hbase, Gary Helmling and Eugene Koontz. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. }; Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. }; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs ----- pom.xml 382c7c2 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 74b9e62 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/#review3355
          -----------------------------------------------------------

          This last version pulls in 3.4? Maybe I'm blind but I don't see it. Mind putting a patch up in JIRA for hadoopqa patch-build to run Andrew? Good stuff.

          • Michael

          On 2011-11-18 16:05:49, Andrew Purtell wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2837/

          -----------------------------------------------------------

          (Updated 2011-11-18 16:05:49)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Summary

          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. };

          Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. };

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"

          HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.

          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs

          -----

          pom.xml 382c7c2

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 74b9e62

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2837/diff

          Testing

          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/#review3355 ----------------------------------------------------------- This last version pulls in 3.4? Maybe I'm blind but I don't see it. Mind putting a patch up in JIRA for hadoopqa patch-build to run Andrew? Good stuff. Michael On 2011-11-18 16:05:49, Andrew Purtell wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-18 16:05:49) Review request for hbase, Gary Helmling and Eugene Koontz. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. }; Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. }; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs ----- pom.xml 382c7c2 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 74b9e62 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-18 16:59:52, Michael Stack wrote:

          > This last version pulls in 3.4? Maybe I'm blind but I don't see it. Mind putting a patch up in JIRA for hadoopqa patch-build to run Andrew? Good stuff.

          Patch won't apply until 3025 is committed.

          • Andrew

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/#review3355
          -----------------------------------------------------------

          On 2011-11-18 16:05:49, Andrew Purtell wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2837/

          -----------------------------------------------------------

          (Updated 2011-11-18 16:05:49)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Summary

          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. };

          Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. };

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"

          HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.

          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs

          -----

          pom.xml 382c7c2

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 74b9e62

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2837/diff

          Testing

          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-18 16:59:52, Michael Stack wrote: > This last version pulls in 3.4? Maybe I'm blind but I don't see it. Mind putting a patch up in JIRA for hadoopqa patch-build to run Andrew? Good stuff. Patch won't apply until 3025 is committed. Andrew ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/#review3355 ----------------------------------------------------------- On 2011-11-18 16:05:49, Andrew Purtell wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-18 16:05:49) Review request for hbase, Gary Helmling and Eugene Koontz. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. }; Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. }; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs ----- pom.xml 382c7c2 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 74b9e62 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-18 04:14:48, Andrew Purtell wrote:

          > src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java, line 338

          > <https://reviews.apache.org/r/2837/diff/3/?file=59201#file59201line338>

          >

          > I don't see how to work around this. The code has to be recompiled against 3.3 or 3.4.

          >

          > Any ideas?

          >

          > I think we have to pull in 3.4 unconditionally.

          Lars Hofhansl wrote:

          Might be a good thing anyway. 3.4 is in RC right now, would probably be GA by the time we GA 0.92. (we're meeting with Ted Dunning tomorrow - unrelated - can ask him what he thinks about it).

          Andrew Purtell wrote:

          We can put in the other two patches and defer this one until 3.4 is released. Consequence would be that 3025 can be subverted if one allows direct client access to the ZK cluster.

          Michael Stack wrote:

          +1 on fixing this patch so it just upgrades our zk to 3.4. I just tried hbase w/ a 3.4 client and a 3.3.3. ensemble and it seems to work. I asked Mahadev to be sure and he says "A 3.3.* client should be able to talk to 3.4.0 server and vice versa." Its as yet unreleased but I'm pretty sure it'll be out before we ship – maybe even a 3.4.1 (smile).

          Andrew Purtell wrote:

          Say what you will about the Hadoop Record compiler aka Jute, but ZK is 1) the only project that uses it that I know of 2) yet consistently can answer in the affirmative regarding cross-version compatibility.

          Let me add, Mahadev tried it himself on his end too and it seemed fine and then he did the caveat that 3.4 has not yet been used in production.....

          • Michael

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/#review3344
          -----------------------------------------------------------

          On 2011-11-18 16:05:49, Andrew Purtell wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2837/

          -----------------------------------------------------------

          (Updated 2011-11-18 16:05:49)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Summary

          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. };

          Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. };

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"

          HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.

          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs

          -----

          pom.xml 382c7c2

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 74b9e62

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2837/diff

          Testing

          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-18 04:14:48, Andrew Purtell wrote: > src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java, line 338 > < https://reviews.apache.org/r/2837/diff/3/?file=59201#file59201line338 > > > I don't see how to work around this. The code has to be recompiled against 3.3 or 3.4. > > Any ideas? > > I think we have to pull in 3.4 unconditionally. Lars Hofhansl wrote: Might be a good thing anyway. 3.4 is in RC right now, would probably be GA by the time we GA 0.92. (we're meeting with Ted Dunning tomorrow - unrelated - can ask him what he thinks about it). Andrew Purtell wrote: We can put in the other two patches and defer this one until 3.4 is released. Consequence would be that 3025 can be subverted if one allows direct client access to the ZK cluster. Michael Stack wrote: +1 on fixing this patch so it just upgrades our zk to 3.4. I just tried hbase w/ a 3.4 client and a 3.3.3. ensemble and it seems to work. I asked Mahadev to be sure and he says "A 3.3.* client should be able to talk to 3.4.0 server and vice versa." Its as yet unreleased but I'm pretty sure it'll be out before we ship – maybe even a 3.4.1 (smile). Andrew Purtell wrote: Say what you will about the Hadoop Record compiler aka Jute, but ZK is 1) the only project that uses it that I know of 2) yet consistently can answer in the affirmative regarding cross-version compatibility. Let me add, Mahadev tried it himself on his end too and it seemed fine and then he did the caveat that 3.4 has not yet been used in production..... Michael ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/#review3344 ----------------------------------------------------------- On 2011-11-18 16:05:49, Andrew Purtell wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-18 16:05:49) Review request for hbase, Gary Helmling and Eugene Koontz. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. }; Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. }; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs ----- pom.xml 382c7c2 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 74b9e62 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-18 16:59:52, Michael Stack wrote:

          > This last version pulls in 3.4? Maybe I'm blind but I don't see it. Mind putting a patch up in JIRA for hadoopqa patch-build to run Andrew? Good stuff.

          Andrew Purtell wrote:

          Patch won't apply until 3025 is committed.

          3025 just went in to 0.92 and trunk, so we should be good to put the patch up for testing.

          • Gary

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/#review3355
          -----------------------------------------------------------

          On 2011-11-18 16:05:49, Andrew Purtell wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2837/

          -----------------------------------------------------------

          (Updated 2011-11-18 16:05:49)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Summary

          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. };

          Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. };

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"

          HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.

          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs

          -----

          pom.xml 382c7c2

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 74b9e62

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2837/diff

          Testing

          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-18 16:59:52, Michael Stack wrote: > This last version pulls in 3.4? Maybe I'm blind but I don't see it. Mind putting a patch up in JIRA for hadoopqa patch-build to run Andrew? Good stuff. Andrew Purtell wrote: Patch won't apply until 3025 is committed. 3025 just went in to 0.92 and trunk, so we should be good to put the patch up for testing. Gary ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/#review3355 ----------------------------------------------------------- On 2011-11-18 16:05:49, Andrew Purtell wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-18 16:05:49) Review request for hbase, Gary Helmling and Eugene Koontz. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. }; Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. }; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs ----- pom.xml 382c7c2 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 74b9e62 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-18 16:59:52, Michael Stack wrote:

          > This last version pulls in 3.4? Maybe I'm blind but I don't see it. Mind putting a patch up in JIRA for hadoopqa patch-build to run Andrew? Good stuff.

          Andrew Purtell wrote:

          Patch won't apply until 3025 is committed.

          Gary Helmling wrote:

          3025 just went in to 0.92 and trunk, so we should be good to put the patch up for testing.

          Rebased patch against current trunk, running tests, will post patch shortly.

          • Andrew

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/#review3355
          -----------------------------------------------------------

          On 2011-11-18 16:05:49, Andrew Purtell wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2837/

          -----------------------------------------------------------

          (Updated 2011-11-18 16:05:49)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Summary

          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. };

          Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. };

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"

          HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.

          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs

          -----

          pom.xml 382c7c2

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 74b9e62

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2837/diff

          Testing

          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-18 16:59:52, Michael Stack wrote: > This last version pulls in 3.4? Maybe I'm blind but I don't see it. Mind putting a patch up in JIRA for hadoopqa patch-build to run Andrew? Good stuff. Andrew Purtell wrote: Patch won't apply until 3025 is committed. Gary Helmling wrote: 3025 just went in to 0.92 and trunk, so we should be good to put the patch up for testing. Rebased patch against current trunk, running tests, will post patch shortly. Andrew ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/#review3355 ----------------------------------------------------------- On 2011-11-18 16:05:49, Andrew Purtell wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-18 16:05:49) Review request for hbase, Gary Helmling and Eugene Koontz. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. }; Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. }; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs ----- pom.xml 382c7c2 src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java 05abeb7 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 74b9e62 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java f613ba9 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/
          -----------------------------------------------------------

          (Updated 2011-11-19 01:36:34.607701)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Changes
          -------

          Rebased to trunk.

          Updated TestZooKeeperACL so it won't break the build if Hadoop is missing HADOOP-7070, but the issue will be logged at WARN in the test output. (-P security selects an artifact that includes it.)

          Summary
          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server

          { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/etc/hbase/conf/hbase.keytab" storeKey=true useTicketCache=false principal="zookeeper/$HOSTNAME"; }

          ;
          Client

          { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true useTicketCache=false keyTab="/etc/hbase/conf/hbase.keytab" principal="hbase/$HOSTNAME"; }

          ;

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="$

          {HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          HBASE_OPTS="${HBASE_OPTS}

          -Dzookeeper.kerberos.removeHostFromPrincipal=true"
          HBASE_OPTS="$

          {HBASE_OPTS}

          -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.
          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs (updated)


          pom.xml eccf41f
          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java fe6f4a5
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 960c9c1
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87
          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java d1b7647
          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2837/diff

          Testing
          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-19 01:36:34.607701) Review request for hbase, Gary Helmling and Eugene Koontz. Changes ------- Rebased to trunk. Updated TestZooKeeperACL so it won't break the build if Hadoop is missing HADOOP-7070 , but the issue will be logged at WARN in the test output. (-P security selects an artifact that includes it.) Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/etc/hbase/conf/hbase.keytab" storeKey=true useTicketCache=false principal="zookeeper/$HOSTNAME"; } ; Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true useTicketCache=false keyTab="/etc/hbase/conf/hbase.keytab" principal="hbase/$HOSTNAME"; } ; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="$ {HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="$ {HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs (updated) pom.xml eccf41f src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java fe6f4a5 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 960c9c1 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java d1b7647 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/#review3368
          -----------------------------------------------------------

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java
          <https://reviews.apache.org/r/2837/#comment7546>

          Missing 'return' here will be added upon commit.

          • Andrew

          On 2011-11-19 01:36:34, Andrew Purtell wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2837/

          -----------------------------------------------------------

          (Updated 2011-11-19 01:36:34)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Summary

          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. };

          Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. };

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"

          HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.

          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs

          -----

          pom.xml eccf41f

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java fe6f4a5

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 960c9c1

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java d1b7647

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2837/diff

          Testing

          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/#review3368 ----------------------------------------------------------- src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java < https://reviews.apache.org/r/2837/#comment7546 > Missing 'return' here will be added upon commit. Andrew On 2011-11-19 01:36:34, Andrew Purtell wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-19 01:36:34) Review request for hbase, Gary Helmling and Eugene Koontz. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. }; Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. }; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs ----- pom.xml eccf41f src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java fe6f4a5 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 960c9c1 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java d1b7647 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          Andrew Purtell added a comment -

          Missing 'return'

          Show
          Andrew Purtell added a comment - Missing 'return'
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12504324/HBASE-2418-5.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 7 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/304//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12504324/HBASE-2418-5.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/304//console This message is automatically generated.
          Hide
          Andrew Purtell added a comment -

          This time again with --no-prefix

          Show
          Andrew Purtell added a comment - This time again with --no-prefix
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-18 04:14:48, Andrew Purtell wrote:

          > src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java, line 338

          > <https://reviews.apache.org/r/2837/diff/3/?file=59201#file59201line338>

          >

          > I don't see how to work around this. The code has to be recompiled against 3.3 or 3.4.

          >

          > Any ideas?

          >

          > I think we have to pull in 3.4 unconditionally.

          Lars Hofhansl wrote:

          Might be a good thing anyway. 3.4 is in RC right now, would probably be GA by the time we GA 0.92. (we're meeting with Ted Dunning tomorrow - unrelated - can ask him what he thinks about it).

          Andrew Purtell wrote:

          We can put in the other two patches and defer this one until 3.4 is released. Consequence would be that 3025 can be subverted if one allows direct client access to the ZK cluster.

          Michael Stack wrote:

          +1 on fixing this patch so it just upgrades our zk to 3.4. I just tried hbase w/ a 3.4 client and a 3.3.3. ensemble and it seems to work. I asked Mahadev to be sure and he says "A 3.3.* client should be able to talk to 3.4.0 server and vice versa." Its as yet unreleased but I'm pretty sure it'll be out before we ship – maybe even a 3.4.1 (smile).

          Michael Stack wrote:

          Let me add, Mahadev tried it himself on his end too and it seemed fine and then he did the caveat that 3.4 has not yet been used in production.....

          Andrew Purtell wrote:

          Say what you will about the Hadoop Record compiler aka Jute, but ZK is 1) the only project that uses it that I know of 2) yet consistently can answer in the affirmative regarding cross-version compatibility.

          Ted said 3.4RC is stable and wire and diskformat compatible with 3.3. Apparently 3.4 will be happening soon.

          • Lars

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2837/#review3344
          -----------------------------------------------------------

          On 2011-11-19 01:36:34, Andrew Purtell wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2837/

          -----------------------------------------------------------

          (Updated 2011-11-19 01:36:34)

          Review request for hbase, Gary Helmling and Eugene Koontz.

          Summary

          -------

          These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control).

          SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example:

          Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. };

          Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. };

          and then configure both the client and server processes to use it, for example in hbase-site.xml:

          HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf"
          bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true"

          HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true"

          HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers.

          There is extraneous whitespace in code surrounding these changes.

          This addresses bug HBASE-2418.

          https://issues.apache.org/jira/browse/HBASE-2418

          Diffs

          -----

          pom.xml eccf41f

          src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java fe6f4a5

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 960c9c1

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87

          src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java d1b7647

          src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2837/diff

          Testing

          -------

          These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0.

          New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk.

          Thanks,

          Andrew

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-18 04:14:48, Andrew Purtell wrote: > src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java, line 338 > < https://reviews.apache.org/r/2837/diff/3/?file=59201#file59201line338 > > > I don't see how to work around this. The code has to be recompiled against 3.3 or 3.4. > > Any ideas? > > I think we have to pull in 3.4 unconditionally. Lars Hofhansl wrote: Might be a good thing anyway. 3.4 is in RC right now, would probably be GA by the time we GA 0.92. (we're meeting with Ted Dunning tomorrow - unrelated - can ask him what he thinks about it). Andrew Purtell wrote: We can put in the other two patches and defer this one until 3.4 is released. Consequence would be that 3025 can be subverted if one allows direct client access to the ZK cluster. Michael Stack wrote: +1 on fixing this patch so it just upgrades our zk to 3.4. I just tried hbase w/ a 3.4 client and a 3.3.3. ensemble and it seems to work. I asked Mahadev to be sure and he says "A 3.3.* client should be able to talk to 3.4.0 server and vice versa." Its as yet unreleased but I'm pretty sure it'll be out before we ship – maybe even a 3.4.1 (smile). Michael Stack wrote: Let me add, Mahadev tried it himself on his end too and it seemed fine and then he did the caveat that 3.4 has not yet been used in production..... Andrew Purtell wrote: Say what you will about the Hadoop Record compiler aka Jute, but ZK is 1) the only project that uses it that I know of 2) yet consistently can answer in the affirmative regarding cross-version compatibility. Ted said 3.4RC is stable and wire and diskformat compatible with 3.3. Apparently 3.4 will be happening soon. Lars ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/#review3344 ----------------------------------------------------------- On 2011-11-19 01:36:34, Andrew Purtell wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2837/ ----------------------------------------------------------- (Updated 2011-11-19 01:36:34) Review request for hbase, Gary Helmling and Eugene Koontz. Summary ------- These changes add support for protecting the state of HBase znodes on a multi-tenant ZooKeeper cluster. This support requires ZK 3.4.0, currently at RC2. It is a companion patch to HBASE-2742 (secure RPC), and HBASE-3025 (Coprocessor based access control). SASL authentication of ZooKeeper clients with the quorum is handled in the ZK client independently of HBase concerns. To enable strong ZK authentication, one must create a suitable JaaS configuration, for example: Server { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. storeKey=true bq. useTicketCache=false bq. principal="zookeeper/$HOSTNAME"; bq. }; Client { bq. com.sun.security.auth.module.Krb5LoginModule required bq. useKeyTab=true bq. useTicketCache=false bq. keyTab="/etc/hbase/conf/hbase.keytab" bq. principal="hbase/$HOSTNAME"; bq. }; and then configure both the client and server processes to use it, for example in hbase-site.xml: HBASE_OPTS="${HBASE_OPTS} -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf" bq. HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeHostFromPrincipal=true" HBASE_OPTS="${HBASE_OPTS} -Dzookeeper.kerberos.removeRealmFromPrincipal=true" HBase will then secure all znodes but for a few world-readable read-only ones needed for clients to look up region locations. All internal cluster operations will be protected from unauthenticated ZK clients, or clients not authenticated to the HBase principal. Presumably the only ZK clients authenticated to the HBase principal will be those embedded in the master and regionservers. There is extraneous whitespace in code surrounding these changes. This addresses bug HBASE-2418 . https://issues.apache.org/jira/browse/HBASE-2418 Diffs ----- pom.xml eccf41f src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java fe6f4a5 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 960c9c1 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java a75cf87 src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java d1b7647 src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PRE-CREATION Diff: https://reviews.apache.org/r/2837/diff Testing ------- These changes are running in production at Trend Micro, using a snapshot build of ZooKeeper 3.4.0. New unit test TestZooKeeperACL passes 100 iterations. All test pass not otherwise currently failing on trunk. Thanks, Andrew
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12504329/HBASE-2418-5.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 7 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 60 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.regionserver.wal.TestLogRolling
          org.apache.hadoop.hbase.client.TestShell
          org.apache.hadoop.hbase.client.TestAdmin

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/305//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/305//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/305//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12504329/HBASE-2418-5.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 60 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.wal.TestLogRolling org.apache.hadoop.hbase.client.TestShell org.apache.hadoop.hbase.client.TestAdmin Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/305//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/305//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/305//console This message is automatically generated.
          Hide
          stack added a comment -

          Could the TestAdmin be because of this patch A?

          Error Message
          
          An error is preventing HBase from connecting to ZooKeeper
          Stacktrace
          
          org.apache.hadoop.hbase.ZooKeeperConnectionException: An error is preventing HBase from connecting to ZooKeeper
          

          (Might be same issue w/ the TestShell fails)

          TestLogRolling seems innocuous.. that can't lock data dir.

          Show
          stack added a comment - Could the TestAdmin be because of this patch A? Error Message An error is preventing HBase from connecting to ZooKeeper Stacktrace org.apache.hadoop.hbase.ZooKeeperConnectionException: An error is preventing HBase from connecting to ZooKeeper (Might be same issue w/ the TestShell fails) TestLogRolling seems innocuous.. that can't lock data dir.
          Hide
          Andrew Purtell added a comment -

          Could the TestAdmin be because of this patch A?

          I don't see it locally.

          This change in the patch could be suspect but it's a shot in the dark:

          --- src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
          +++ src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
          @@ -148,11 +156,14 @@ public class MiniZooKeeperCluster {
                   tickTimeToUse = TICK_TIME;
                 }
                 ZooKeeperServer server = new ZooKeeperServer(dir, dir, tickTimeToUse);
          -      NIOServerCnxn.Factory standaloneServerFactory;
          +      NIOServerCnxnFactory standaloneServerFactory;
                 while (true) {
                   try {
          -          standaloneServerFactory = new NIOServerCnxn.Factory(
          -              new InetSocketAddress(tentativePort));
          +          standaloneServerFactory = new NIOServerCnxnFactory();
          +          standaloneServerFactory.configure(
          +            new InetSocketAddress(tentativePort),
          +            configuration.getInt(HConstants.ZOOKEEPER_MAX_CLIENT_CNXNS,
          +              HConstants.DEFAULT_ZOOKEPER_MAX_CLIENT_CNXNS));
                   } catch (BindException e) {
                     LOG.debug("Failed binding ZK Server to client port: " +
                         tentativePort);
          

          I could change HConstants.DEFAULT_ZOOKEPER_MAX_CLIENT_CNXNS here to 1000 and resubmit.

          Show
          Andrew Purtell added a comment - Could the TestAdmin be because of this patch A? I don't see it locally. This change in the patch could be suspect but it's a shot in the dark: --- src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java +++ src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java @@ -148,11 +156,14 @@ public class MiniZooKeeperCluster { tickTimeToUse = TICK_TIME; } ZooKeeperServer server = new ZooKeeperServer(dir, dir, tickTimeToUse); - NIOServerCnxn.Factory standaloneServerFactory; + NIOServerCnxnFactory standaloneServerFactory; while ( true ) { try { - standaloneServerFactory = new NIOServerCnxn.Factory( - new InetSocketAddress(tentativePort)); + standaloneServerFactory = new NIOServerCnxnFactory(); + standaloneServerFactory.configure( + new InetSocketAddress(tentativePort), + configuration.getInt(HConstants.ZOOKEEPER_MAX_CLIENT_CNXNS, + HConstants.DEFAULT_ZOOKEPER_MAX_CLIENT_CNXNS)); } catch (BindException e) { LOG.debug( "Failed binding ZK Server to client port: " + tentativePort); I could change HConstants.DEFAULT_ZOOKEPER_MAX_CLIENT_CNXNS here to 1000 and resubmit.
          Hide
          Andrew Purtell added a comment -

          v6 patch with above described change.

          Show
          Andrew Purtell added a comment - v6 patch with above described change.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12504379/HBASE-2418-6.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 7 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/312//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12504379/HBASE-2418-6.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/312//console This message is automatically generated.
          Hide
          Andrew Purtell added a comment -

          Rebased patch on latest trunk.

          Show
          Andrew Purtell added a comment - Rebased patch on latest trunk.
          Hide
          stack added a comment -

          Trunk is changing too fast on you Andrew!

          patching file pom.xml
          Hunk #1 FAILED at 276.
          Hunk #2 succeeded at 861 (offset 41 lines).
          Hunk #3 succeeded at 1404 with fuzz 2 (offset 3 lines).
          1 out of 3 hunks FAILED -- saving rejects to file pom.xml.rej
          patching file src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
          patching file src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          patching file src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
          patching file src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
          patching file src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java
          PATCH APPLICATION FAILED
          

          This is last 0.92 patch though.... Almost there.

          Show
          stack added a comment - Trunk is changing too fast on you Andrew! patching file pom.xml Hunk #1 FAILED at 276. Hunk #2 succeeded at 861 (offset 41 lines). Hunk #3 succeeded at 1404 with fuzz 2 (offset 3 lines). 1 out of 3 hunks FAILED -- saving rejects to file pom.xml.rej patching file src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java patching file src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java patching file src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java patching file src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java patching file src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java PATCH APPLICATION FAILED This is last 0.92 patch though.... Almost there.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12504384/HBASE-2418-6.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 7 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 60 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.client.TestAdmin
          org.apache.hadoop.hbase.replication.TestReplication
          org.apache.hadoop.hbase.client.TestShell

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/313//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/313//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12504384/HBASE-2418-6.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 60 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestAdmin org.apache.hadoop.hbase.replication.TestReplication org.apache.hadoop.hbase.client.TestShell Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/313//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/313//console This message is automatically generated.
          Hide
          Andrew Purtell added a comment -

          Committed to trunk and 0.92.

          TestZooKeeperACL passes with and without '-P security' locally. Does not break the build if '-P security' is not specified. Test failures found by HudsonQA are not directly related to this change.

          Show
          Andrew Purtell added a comment - Committed to trunk and 0.92. TestZooKeeperACL passes with and without '-P security' locally. Does not break the build if '-P security' is not specified. Test failures found by HudsonQA are not directly related to this change.
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #2466 (See https://builds.apache.org/job/HBase-TRUNK/2466/)
          HBASE-2418 Support for ZooKeeper authentication

          apurtell :
          Files :

          • /hbase/trunk/pom.xml
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK #2466 (See https://builds.apache.org/job/HBase-TRUNK/2466/ ) HBASE-2418 Support for ZooKeeper authentication apurtell : Files : /hbase/trunk/pom.xml /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java
          Hide
          Hudson added a comment -

          Integrated in HBase-0.92 #152 (See https://builds.apache.org/job/HBase-0.92/152/)
          HBASE-2418 Support for ZooKeeper authentication

          apurtell :
          Files :

          • /hbase/branches/0.92/CHANGES.txt
          • /hbase/branches/0.92/pom.xml
          • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
          • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
          • /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
          • /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java
          Show
          Hudson added a comment - Integrated in HBase-0.92 #152 (See https://builds.apache.org/job/HBase-0.92/152/ ) HBASE-2418 Support for ZooKeeper authentication apurtell : Files : /hbase/branches/0.92/CHANGES.txt /hbase/branches/0.92/pom.xml /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java
          Hide
          Ted Yu added a comment -

          Addendum adds Gary's maven repository to pom

          Show
          Ted Yu added a comment - Addendum adds Gary's maven repository to pom
          Hide
          Ted Yu added a comment -

          Applied addendum to 0.92 branch.
          Build 153 is running tests as this moment.

          Show
          Ted Yu added a comment - Applied addendum to 0.92 branch. Build 153 is running tests as this moment.
          Hide
          Hudson added a comment -

          Integrated in HBase-0.92 #153 (See https://builds.apache.org/job/HBase-0.92/153/)
          HBASE-2418 Addendum adds Gary's maven repo to pom.xml

          tedyu :
          Files :

          • /hbase/branches/0.92/pom.xml
          Show
          Hudson added a comment - Integrated in HBase-0.92 #153 (See https://builds.apache.org/job/HBase-0.92/153/ ) HBASE-2418 Addendum adds Gary's maven repo to pom.xml tedyu : Files : /hbase/branches/0.92/pom.xml
          Hide
          Andrew Purtell added a comment -

          Thanks Ted. I thought that went in with HBASE-3025.

          Show
          Andrew Purtell added a comment - Thanks Ted. I thought that went in with HBASE-3025 .
          Hide
          Andrew Purtell added a comment -

          And it looks like this part of the POM in trunk is not in the POM on 0.92:

            <pluginRepositories>
              <pluginRepository>
                <id>ghelmling.testing</id>
                <name>Gary Helmling test repo</name>
                <url>http://people.apache.org/~garyh/mvn/</url>
                <snapshots>
                  <enabled>true</enabled>
                </snapshots>
                <releases>
                  <enabled>true</enabled>
                </releases>
              </pluginRepository>
            </pluginRepositories>
          

          I don't know enough about Maven or how Gary set up the security profile to know if it is needed or not. Gary?

          Show
          Andrew Purtell added a comment - And it looks like this part of the POM in trunk is not in the POM on 0.92: <pluginRepositories> <pluginRepository> <id>ghelmling.testing</id> <name>Gary Helmling test repo</name> <url>http: //people.apache.org/~garyh/mvn/</url> <snapshots> <enabled> true </enabled> </snapshots> <releases> <enabled> true </enabled> </releases> </pluginRepository> </pluginRepositories> I don't know enough about Maven or how Gary set up the security profile to know if it is needed or not. Gary?
          Hide
          Hudson added a comment -

          Integrated in HBase-0.92-security #2 (See https://builds.apache.org/job/HBase-0.92-security/2/)
          HBASE-2418 Addendum adds Gary's maven repo to pom.xml
          HBASE-2418 Support for ZooKeeper authentication

          tedyu :
          Files :

          • /hbase/branches/0.92/pom.xml

          apurtell :
          Files :

          • /hbase/branches/0.92/CHANGES.txt
          • /hbase/branches/0.92/pom.xml
          • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
          • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
          • /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
          • /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java
          Show
          Hudson added a comment - Integrated in HBase-0.92-security #2 (See https://builds.apache.org/job/HBase-0.92-security/2/ ) HBASE-2418 Addendum adds Gary's maven repo to pom.xml HBASE-2418 Support for ZooKeeper authentication tedyu : Files : /hbase/branches/0.92/pom.xml apurtell : Files : /hbase/branches/0.92/CHANGES.txt /hbase/branches/0.92/pom.xml /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java
          Hide
          Andrew Purtell added a comment -

          Hudson returned another build failure report. I committed the above to the 0.92 POM.

          Show
          Andrew Purtell added a comment - Hudson returned another build failure report. I committed the above to the 0.92 POM.
          Hide
          Hudson added a comment -

          Integrated in HBase-0.92 #154 (See https://builds.apache.org/job/HBase-0.92/154/)
          Amend HBASE-2418 Add pluginRepositories to POM

          apurtell :
          Files :

          • /hbase/branches/0.92/pom.xml
          Show
          Hudson added a comment - Integrated in HBase-0.92 #154 (See https://builds.apache.org/job/HBase-0.92/154/ ) Amend HBASE-2418 Add pluginRepositories to POM apurtell : Files : /hbase/branches/0.92/pom.xml
          Hide
          Gary Helmling added a comment -

          The <pluginRepositories/> entry was added for HBASE-4763/HBASE-4781 for the custom maven-surefire build. It's not needed for the security components and should not be in the 0.92 branch as far as I can tell (HBASE-4781 is marked for 0.94).

          Show
          Gary Helmling added a comment - The <pluginRepositories/> entry was added for HBASE-4763 / HBASE-4781 for the custom maven-surefire build. It's not needed for the security components and should not be in the 0.92 branch as far as I can tell ( HBASE-4781 is marked for 0.94).
          Hide
          Hudson added a comment -

          Integrated in HBase-0.92-security #3 (See https://builds.apache.org/job/HBase-0.92-security/3/)
          Amend HBASE-2418 Add pluginRepositories to POM

          apurtell :
          Files :

          • /hbase/branches/0.92/pom.xml
          Show
          Hudson added a comment - Integrated in HBase-0.92-security #3 (See https://builds.apache.org/job/HBase-0.92-security/3/ ) Amend HBASE-2418 Add pluginRepositories to POM apurtell : Files : /hbase/branches/0.92/pom.xml
          Hide
          Gary Helmling added a comment -

          http://monitoring.apache.org/status/ is showing people.apache.org is down (minotaur.apache.org). This is probably the cause of the build failures, which are showing connection timed out retrieving artifacts from my repo.

          Show
          Gary Helmling added a comment - http://monitoring.apache.org/status/ is showing people.apache.org is down (minotaur.apache.org). This is probably the cause of the build failures, which are showing connection timed out retrieving artifacts from my repo.
          Hide
          Mikhail Bautin added a comment -

          I just saw this regionserver crash in my five-node, three-RS cluster test. Since this is a ZK-related patch that went in recently, I am attaching the stack trace here just in case.

          2011-11-21 01:30:15,188 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server <machine_name>,60020,1321867814890: Initialization of RS failed. Hence aborting RS.
          java.util.ConcurrentModificationException
          at java.util.Hashtable$Enumerator.next(Hashtable.java:1031)
          at org.apache.hadoop.conf.Configuration.iterator(Configuration.java:1042)
          at org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKProps(ZKConfig.java:75)
          at org.apache.hadoop.hbase.zookeeper.ZKConfig.getZKQuorumServersString(ZKConfig.java:245)
          at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:144)
          at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:124)
          at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:1262)
          at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.setupZookeeperTrackers(HConnectionManager.java:568)
          at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:559)
          at org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:183)
          at org.apache.hadoop.hbase.catalog.CatalogTracker.<init>(CatalogTracker.java:177)
          at org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:575)
          at org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:534)
          at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:642)
          at java.lang.Thread.run(Thread.java:619)

          Show
          Mikhail Bautin added a comment - I just saw this regionserver crash in my five-node, three-RS cluster test. Since this is a ZK-related patch that went in recently, I am attaching the stack trace here just in case. 2011-11-21 01:30:15,188 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server <machine_name>,60020,1321867814890: Initialization of RS failed. Hence aborting RS. java.util.ConcurrentModificationException at java.util.Hashtable$Enumerator.next(Hashtable.java:1031) at org.apache.hadoop.conf.Configuration.iterator(Configuration.java:1042) at org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKProps(ZKConfig.java:75) at org.apache.hadoop.hbase.zookeeper.ZKConfig.getZKQuorumServersString(ZKConfig.java:245) at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:144) at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:124) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:1262) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.setupZookeeperTrackers(HConnectionManager.java:568) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:559) at org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:183) at org.apache.hadoop.hbase.catalog.CatalogTracker.<init>(CatalogTracker.java:177) at org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:575) at org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:534) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:642) at java.lang.Thread.run(Thread.java:619)
          Hide
          ramkrishna.s.vasudevan added a comment -

          Not able to build from the maven repository for zookeeper 3.4.0 SNAPSHOT.
          Correct me if am wrong.

          Show
          ramkrishna.s.vasudevan added a comment - Not able to build from the maven repository for zookeeper 3.4.0 SNAPSHOT. Correct me if am wrong.
          Hide
          ramkrishna.s.vasudevan added a comment -

          I resolved by adding this

          <repository>
                <id>ghelmling.testing</id>
                <name>Gary Helmling test repo</name>
                <url>http://people.apache.org/~garyh/mvn/</url>
                <snapshots>
                  <enabled>true</enabled>
                </snapshots>
                <releases>
                  <enabled>true</enabled>
                </releases>
              </repository>
          

          this was present in HBASE-2418-3.patch

          Show
          ramkrishna.s.vasudevan added a comment - I resolved by adding this <repository> <id>ghelmling.testing</id> <name>Gary Helmling test repo</name> <url>http: //people.apache.org/~garyh/mvn/</url> <snapshots> <enabled> true </enabled> </snapshots> <releases> <enabled> true </enabled> </releases> </repository> this was present in HBASE-2418 -3.patch
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK-security #2 (See https://builds.apache.org/job/HBase-TRUNK-security/2/)
          HBASE-2418 Support for ZooKeeper authentication

          apurtell :
          Files :

          • /hbase/trunk/pom.xml
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK-security #2 (See https://builds.apache.org/job/HBase-TRUNK-security/2/ ) HBASE-2418 Support for ZooKeeper authentication apurtell : Files : /hbase/trunk/pom.xml /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java
          Hide
          Andrew Purtell added a comment -

          @Mikhail: Thanks, that doesn't have a clear direct relation. If it were a test failure, I'd say otherwise. This patch modified the MiniZKCluster to take a Configuration in constructor and use it. This patch did not touch ZKConfig, which is HBase side code.

          Show
          Andrew Purtell added a comment - @Mikhail: Thanks, that doesn't have a clear direct relation. If it were a test failure, I'd say otherwise. This patch modified the MiniZKCluster to take a Configuration in constructor and use it. This patch did not touch ZKConfig, which is HBase side code.
          Hide
          Andrew Purtell added a comment -

          @Ram I'm looking at the 0.92 pom right now and it includes the repository entry for ghelmling.testing.

          Show
          Andrew Purtell added a comment - @Ram I'm looking at the 0.92 pom right now and it includes the repository entry for ghelmling.testing.
          Hide
          Andrew Purtell added a comment -

          I opened HBASE-4835 for the CME.

          Show
          Andrew Purtell added a comment - I opened HBASE-4835 for the CME.
          Hide
          Eugene Koontz added a comment -

          HBase clients can authenticate with Kerberos using kinit and connect to hbase (e.g. with hbase shell) with -Djava.security.auth.login.config=/path/to/client/jaas.conf , where this configuration file is:

          Client {
           com.sun.security.auth.module.Krb5LoginModule required
           useKeyTab=false
           useTicketCache=true
           doNotPrompt=true
           renewTGT=true;
          };
          
          Show
          Eugene Koontz added a comment - HBase clients can authenticate with Kerberos using kinit and connect to hbase (e.g. with hbase shell) with -Djava.security.auth.login.config=/path/to/client/jaas.conf , where this configuration file is: Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab= false useTicketCache= true doNotPrompt= true renewTGT= true ; };
          Hide
          stack added a comment -

          @Eugene Should your above comment go into the hbase book? i.e. on how to set up secure zk?

          Show
          stack added a comment - @Eugene Should your above comment go into the hbase book? i.e. on how to set up secure zk?
          Hide
          Eugene Koontz added a comment -

          @stack, Great idea. Will learn more about how to add content to the hbase book and will create a patch and JIRA.

          Show
          Eugene Koontz added a comment - @stack, Great idea. Will learn more about how to add content to the hbase book and will create a patch and JIRA.
          Hide
          stack added a comment -

          If you write up a bit of text, I can add it np E.

          Show
          stack added a comment - If you write up a bit of text, I can add it np E.
          Hide
          Eugene Koontz added a comment -

          HBASE-4376 addresses need for security-related documentation, as also discussed recently in this JIRA.

          Show
          Eugene Koontz added a comment - HBASE-4376 addresses need for security-related documentation, as also discussed recently in this JIRA.
          Hide
          Eugene Koontz added a comment -

          Hi @stack, thanks for encouraging me to write some documentation relating to this; please see: https://issues.apache.org/jira/browse/HBASE-4960

          Show
          Eugene Koontz added a comment - Hi @stack, thanks for encouraging me to write some documentation relating to this; please see: https://issues.apache.org/jira/browse/HBASE-4960

            People

            • Assignee:
              Eugene Koontz
              Reporter:
              Patrick Hunt
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development