Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-6523

Support for HBase Registry Implementations through Phoenix connection URL

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 5.2.0, 5.1.4
    • core
    • None
    • Hide
      Add support for MasterRegistry and RPCConnectionRegistry to Phoenix.

      Introduces the new URL protocol variants:
      * jdbc:phoenix+zk: Uses Zookeeper. This is the original registry supported since the inception of HBase and Phoenix.
      * jdbc:phoenix+rpc: Uses RPC to connecto to the specified HBase RS/Master nodes.
      * jdbc:phoenix+master: Uses RPC to connect to the specified HBase Master nodes

      The syntax:
      "jdbc:phoenix" : uses the default registry and and connection from hbase-site.xml

      "jdbc:phoenix:param1:param2...": Protocol/Registry is determined from Hbase version and hbase-site.xml configuration, and parameters are interpreted accoring to the registry.

      "jdbc:phoenix+zk:hosts:ports:zknode:principal:keytab;options..." : Behaves the same as jdbc:phoenix... URL previously. Any missing parameters use defaults fom hbase-site.xml or the environment.

      "jdbc:phoenix+rpc:hosts:ports::principal:keytab;options..." : Uses RPCConnectionRegistry. If more than two options are specified, then the third one (he unused zkNode paramater) must always be blank.

      "jdbc:phoenix+master:hosts:ports::principal:keytab;options..." : Uses RPCMasterRegistry. If more than two options are specified, then the third one (he unused zkNode paramater) must always be blank.

      Phoenix now also supports heterogenous ports defined in HBASE-12706 for every registry. When specifying the ports for each host separately the colon ":" character must be escaped with a backslash, i.e. "jdbc:phoenix+zk:host1\:123,host2\:345:/hbase:principal:keytab", or "jdbc:phoenix+rpc:host1\:123,host2\:345" You may need to add extra escapes to preserve the backslashes if defined in java code, etc.

      Note that while the phoenix+zk URL handling code has heuristics that tries to handle some omitted parameters, the Master and ConnectionRPC registry code strictly maps the URL parameters to by their ordering.

      Note that Phoenix now internally normalizes the URL. Whether you specify an explicit connection, or use the default "jdbc:phoenix" URL, Phoenix will internally normalize the connection, and set the properties for the internal HBase Connection objects appropriately.

      Also note that for most non-HA use cases an explicit connection URL should NOT be used. The preferred way to specify the connection is to have an up-to-date hbase-site.xml with both Hbase and Phoenix client properties set correctly (with other Hadoop conficguration files as needed) on the Phoenix application classpath , and using the default "jdbc:phoenix" URL.
      Show
      Add support for MasterRegistry and RPCConnectionRegistry to Phoenix. Introduces the new URL protocol variants: * jdbc:phoenix+zk: Uses Zookeeper. This is the original registry supported since the inception of HBase and Phoenix. * jdbc:phoenix+rpc: Uses RPC to connecto to the specified HBase RS/Master nodes. * jdbc:phoenix+master: Uses RPC to connect to the specified HBase Master nodes The syntax: "jdbc:phoenix" : uses the default registry and and connection from hbase-site.xml "jdbc:phoenix:param1:param2...": Protocol/Registry is determined from Hbase version and hbase-site.xml configuration, and parameters are interpreted accoring to the registry. "jdbc:phoenix+zk:hosts:ports:zknode:principal:keytab;options..." : Behaves the same as jdbc:phoenix... URL previously. Any missing parameters use defaults fom hbase-site.xml or the environment. "jdbc:phoenix+rpc:hosts:ports::principal:keytab;options..." : Uses RPCConnectionRegistry. If more than two options are specified, then the third one (he unused zkNode paramater) must always be blank. "jdbc:phoenix+master:hosts:ports::principal:keytab;options..." : Uses RPCMasterRegistry. If more than two options are specified, then the third one (he unused zkNode paramater) must always be blank. Phoenix now also supports heterogenous ports defined in HBASE-12706 for every registry. When specifying the ports for each host separately the colon ":" character must be escaped with a backslash, i.e. "jdbc:phoenix+zk:host1\:123,host2\:345:/hbase:principal:keytab", or "jdbc:phoenix+rpc:host1\:123,host2\:345" You may need to add extra escapes to preserve the backslashes if defined in java code, etc. Note that while the phoenix+zk URL handling code has heuristics that tries to handle some omitted parameters, the Master and ConnectionRPC registry code strictly maps the URL parameters to by their ordering. Note that Phoenix now internally normalizes the URL. Whether you specify an explicit connection, or use the default "jdbc:phoenix" URL, Phoenix will internally normalize the connection, and set the properties for the internal HBase Connection objects appropriately. Also note that for most non-HA use cases an explicit connection URL should NOT be used. The preferred way to specify the connection is to have an up-to-date hbase-site.xml with both Hbase and Phoenix client properties set correctly (with other Hadoop conficguration files as needed) on the Phoenix application classpath , and using the default "jdbc:phoenix" URL.

    Description

      https://issues.apache.org/jira/browse/HBASE-23305

      https://issues.apache.org/jira/browse/HBASE-18095

       

      HBase now supports a zookeeper-less connection strategy using a Master Registry implementation. 

       

      For this to work, the client simply needs to set a list of <host:port>s of the HMaster quorum

       

      <property>
   <name>hbase.masters</name>
   <value>master1:16000,master2:16001,master3:16000</value>
</property>

       

      To support opting into this from a Phoenix connection URL, we can introduce a "connector type". We'll leverage the + char of JDBC URL grammar to specify the connection type. Connections will start to look something like this:

      jdbc:phoenix+zk:hostname1,2,3...:<properties> 
jdbc:phoenix+hrpc:hostname1,2,3...:<properties>
jdbc:phoenix+bigtable:hostname1,2,3...:<properties>

      Above are examples of opting into hrpc/zk/bigtable registry implementations of HBase.

       

      If no connector is specified, the driver will default to a Zookeeper based connection.

       

       

       

      Attachments

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. PHOENIX-7150 PHOENIX-6523 has broken HBase 2.1 and 2.2 support

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. PHOENIX-7191 Connectionless CQSs don't work with non-ZK registries

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved
          depends upon

          New Feature - A new feature of the product, which has yet to be developed. PHOENIX-6633 Use standard JDBC URL in Spark Connector and make it optional

          • Major - Major loss of function.
          • Resolved
          is a child of

          Sub-task - The sub-task of the issue HBASE-23305 Master based registry implementation

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. HBASE-18095 Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client

          • Major - Major loss of function.
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-12706 Support multiple port numbers in ZK quorum string

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. PHOENIX-7053 Use full JDBC URL in Pherf instead of ZK quorum

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. PHOENIX-7097 Allow specifying full JDBC URL string in psql/PhoenixRuntime and sqllline.py

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #1280

          Web Link GitHub Pull Request #1658

          Web Link GitHub Pull Request #1683

          Web Link GitHub Pull Request #1730

          Web Link Stoty PR

          Activity

            People

              stoty Istvan Toth
              rxmie Ramie Raufdeen
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: